2026-02-17T11:20:29.274 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-17T11:20:29.288 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T11:20:29.364 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:10:36-rados-wip-bharath20-testing-2026-02-13-0906-squid-distro-default-trial/51298 branch: wip-bharath20-testing-2026-02-13-0906-squid description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} email: skanta@redhat.com first_in_suite: false flavor: default job_id: '51298' kernel: branch: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: trial name: skanta-2026-02-17_04:10:36-rados-wip-bharath20-testing-2026-02-13-0906-squid-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-bharath20-testing-2026-02-13-0906-squid ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 3c96dfb3f50b3d02598a82af7104764bfdcd3919 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 3c96dfb3f50b3d02598a82af7104764bfdcd3919 workunit: branch: wip-bharath20-testing-2026-02-13-0906-squid sha1: 3c96dfb3f50b3d02598a82af7104764bfdcd3919 owner: scheduled_skanta@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 4581 sha1: 3c96dfb3f50b3d02598a82af7104764bfdcd3919 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath20-testing-2026-02-13-0906-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_3c96dfb3f50b3d02598a82af7104764bfdcd3919/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 3c96dfb3f50b3d02598a82af7104764bfdcd3919 targets: trial128.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIQ83p1XxO2ylWY8ipPYW1RxyJ17+infdepfqudx80rpRO7KDHFxGI8mTPzwiqea+G5/7RzCiHAAGso5RKsK2GE= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - scrub teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-17_04:10:36 tube: trial user: skanta verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3682 2026-02-17T11:20:29.364 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_3c96dfb3f50b3d02598a82af7104764bfdcd3919/qa; will attempt to use it 2026-02-17T11:20:29.365 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_3c96dfb3f50b3d02598a82af7104764bfdcd3919/qa/tasks 2026-02-17T11:20:29.365 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-17T11:20:29.366 INFO:teuthology.task.internal:Checking packages... 2026-02-17T11:20:29.366 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '3c96dfb3f50b3d02598a82af7104764bfdcd3919' 2026-02-17T11:20:29.366 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-17T11:20:29.366 INFO:teuthology.packaging:ref: None 2026-02-17T11:20:29.366 INFO:teuthology.packaging:tag: None 2026-02-17T11:20:29.366 INFO:teuthology.packaging:branch: wip-bharath20-testing-2026-02-13-0906-squid 2026-02-17T11:20:29.366 INFO:teuthology.packaging:sha1: 3c96dfb3f50b3d02598a82af7104764bfdcd3919 2026-02-17T11:20:29.366 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-bharath20-testing-2026-02-13-0906-squid 2026-02-17T11:20:29.489 INFO:teuthology.task.internal:Found packages for ceph version 19.2.3-675-g3c96dfb3-1jammy 2026-02-17T11:20:29.491 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-17T11:20:29.506 INFO:teuthology.task.internal:no buildpackages task found 2026-02-17T11:20:29.506 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-17T11:20:29.519 INFO:teuthology.task.internal:Saving configuration 2026-02-17T11:20:29.528 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-17T11:20:29.540 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-17T11:20:29.619 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial128.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:10:36-rados-wip-bharath20-testing-2026-02-13-0906-squid-distro-default-trial/51298', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-17 11:17:02.890995', 'locked_by': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFJvb6MhLMual8gUPCtXocL+CC4GVFNWqxqK0hWb/jAP'} 2026-02-17T11:20:29.619 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-17T11:20:29.631 INFO:teuthology.task.internal:roles: ubuntu@trial128.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-17T11:20:29.631 INFO:teuthology.run_tasks:Running task console_log... 2026-02-17T11:20:29.715 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe34e7ca480>, signals=[15]) 2026-02-17T11:20:29.715 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-17T11:20:29.726 INFO:teuthology.task.internal:Opening connections... 2026-02-17T11:20:29.726 DEBUG:teuthology.task.internal:connecting to ubuntu@trial128.front.sepia.ceph.com 2026-02-17T11:20:29.727 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial128.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T11:20:29.790 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-17T11:20:29.805 DEBUG:teuthology.orchestra.run.trial128:> uname -m 2026-02-17T11:20:29.868 INFO:teuthology.orchestra.run.trial128.stdout:x86_64 2026-02-17T11:20:29.869 DEBUG:teuthology.orchestra.run.trial128:> cat /etc/os-release 2026-02-17T11:20:29.912 INFO:teuthology.orchestra.run.trial128.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-17T11:20:29.912 INFO:teuthology.orchestra.run.trial128.stdout:NAME="Ubuntu" 2026-02-17T11:20:29.912 INFO:teuthology.orchestra.run.trial128.stdout:VERSION_ID="22.04" 2026-02-17T11:20:29.913 INFO:teuthology.orchestra.run.trial128.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-17T11:20:29.913 INFO:teuthology.orchestra.run.trial128.stdout:VERSION_CODENAME=jammy 2026-02-17T11:20:29.913 INFO:teuthology.orchestra.run.trial128.stdout:ID=ubuntu 2026-02-17T11:20:29.913 INFO:teuthology.orchestra.run.trial128.stdout:ID_LIKE=debian 2026-02-17T11:20:29.913 INFO:teuthology.orchestra.run.trial128.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-17T11:20:29.913 INFO:teuthology.orchestra.run.trial128.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-17T11:20:29.913 INFO:teuthology.orchestra.run.trial128.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-17T11:20:29.913 INFO:teuthology.orchestra.run.trial128.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-17T11:20:29.913 INFO:teuthology.orchestra.run.trial128.stdout:UBUNTU_CODENAME=jammy 2026-02-17T11:20:29.914 INFO:teuthology.lock.ops:Updating trial128.front.sepia.ceph.com on lock server 2026-02-17T11:20:30.006 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-17T11:20:30.018 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-17T11:20:30.033 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-17T11:20:30.033 DEBUG:teuthology.orchestra.run.trial128:> test '!' -e /home/ubuntu/cephtest 2026-02-17T11:20:30.037 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-17T11:20:30.047 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-17T11:20:30.048 DEBUG:teuthology.orchestra.run.trial128:> test -z $(ls -A /var/lib/ceph) 2026-02-17T11:20:30.085 INFO:teuthology.orchestra.run.trial128.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-17T11:20:30.086 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-17T11:20:30.184 INFO:teuthology.run_tasks:Running task kernel... 2026-02-17T11:20:30.204 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-17T11:20:30.204 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-17T11:20:30.204 DEBUG:teuthology.orchestra.run.trial128:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-17T11:20:30.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T11:20:30.207 DEBUG:teuthology.orchestra.run.trial128:> uname -r 2026-02-17T11:20:30.252 INFO:teuthology.orchestra.run.trial128.stdout:5.15.0-164-generic 2026-02-17T11:20:30.252 INFO:teuthology.task.kernel:Running kernel on trial128: 5.15.0-164-generic 2026-02-17T11:20:30.252 DEBUG:teuthology.orchestra.run.trial128:> sudo apt-get clean 2026-02-17T11:20:30.310 DEBUG:teuthology.orchestra.run.trial128:> sudo apt-get update 2026-02-17T11:20:30.829 INFO:teuthology.orchestra.run.trial128.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-17T11:20:30.922 INFO:teuthology.orchestra.run.trial128.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-17T11:20:31.274 INFO:teuthology.orchestra.run.trial128.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-17T11:20:31.429 INFO:teuthology.orchestra.run.trial128.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-17T11:20:31.574 INFO:teuthology.orchestra.run.trial128.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3236 kB] 2026-02-17T11:20:31.989 INFO:teuthology.orchestra.run.trial128.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [493 kB] 2026-02-17T11:20:32.001 INFO:teuthology.orchestra.run.trial128.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.1 kB] 2026-02-17T11:20:32.002 INFO:teuthology.orchestra.run.trial128.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1255 kB] 2026-02-17T11:20:32.038 INFO:teuthology.orchestra.run.trial128.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [313 kB] 2026-02-17T11:20:32.045 INFO:teuthology.orchestra.run.trial128.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.3 kB] 2026-02-17T11:20:32.046 INFO:teuthology.orchestra.run.trial128.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5287 kB] 2026-02-17T11:20:32.208 INFO:teuthology.orchestra.run.trial128.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [997 kB] 2026-02-17T11:20:32.231 INFO:teuthology.orchestra.run.trial128.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2966 kB] 2026-02-17T11:20:32.320 INFO:teuthology.orchestra.run.trial128.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [426 kB] 2026-02-17T11:20:32.331 INFO:teuthology.orchestra.run.trial128.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-17T11:20:32.332 INFO:teuthology.orchestra.run.trial128.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1017 kB] 2026-02-17T11:20:32.367 INFO:teuthology.orchestra.run.trial128.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe Translation-en [224 kB] 2026-02-17T11:20:32.374 INFO:teuthology.orchestra.run.trial128.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.6 kB] 2026-02-17T11:20:32.374 INFO:teuthology.orchestra.run.trial128.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5096 kB] 2026-02-17T11:20:32.527 INFO:teuthology.orchestra.run.trial128.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [964 kB] 2026-02-17T11:20:35.772 INFO:teuthology.orchestra.run.trial128.stdout:Fetched 22.7 MB in 3s (7691 kB/s) 2026-02-17T11:20:36.295 INFO:teuthology.orchestra.run.trial128.stdout:Reading package lists... 2026-02-17T11:20:36.308 DEBUG:teuthology.orchestra.run.trial128:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-17T11:20:36.342 INFO:teuthology.orchestra.run.trial128.stdout:Reading package lists... 2026-02-17T11:20:36.453 INFO:teuthology.orchestra.run.trial128.stdout:Building dependency tree... 2026-02-17T11:20:36.453 INFO:teuthology.orchestra.run.trial128.stdout:Reading state information... 2026-02-17T11:20:36.538 INFO:teuthology.orchestra.run.trial128.stdout:The following packages were automatically installed and are no longer required: 2026-02-17T11:20:36.538 INFO:teuthology.orchestra.run.trial128.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-17T11:20:36.538 INFO:teuthology.orchestra.run.trial128.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-17T11:20:36.539 INFO:teuthology.orchestra.run.trial128.stdout: sg3-utils-udev 2026-02-17T11:20:36.539 INFO:teuthology.orchestra.run.trial128.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-17T11:20:36.539 INFO:teuthology.orchestra.run.trial128.stdout:The following additional packages will be installed: 2026-02-17T11:20:36.539 INFO:teuthology.orchestra.run.trial128.stdout: linux-generic linux-headers-5.15.0-170 linux-headers-5.15.0-170-generic 2026-02-17T11:20:36.539 INFO:teuthology.orchestra.run.trial128.stdout: linux-headers-generic linux-image-5.15.0-170-generic 2026-02-17T11:20:36.539 INFO:teuthology.orchestra.run.trial128.stdout: linux-modules-5.15.0-170-generic linux-modules-extra-5.15.0-170-generic 2026-02-17T11:20:36.539 INFO:teuthology.orchestra.run.trial128.stdout:Suggested packages: 2026-02-17T11:20:36.539 INFO:teuthology.orchestra.run.trial128.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2026-02-17T11:20:36.552 INFO:teuthology.orchestra.run.trial128.stdout:The following NEW packages will be installed: 2026-02-17T11:20:36.552 INFO:teuthology.orchestra.run.trial128.stdout: linux-headers-5.15.0-170 linux-headers-5.15.0-170-generic 2026-02-17T11:20:36.552 INFO:teuthology.orchestra.run.trial128.stdout: linux-image-5.15.0-170-generic linux-modules-5.15.0-170-generic 2026-02-17T11:20:36.552 INFO:teuthology.orchestra.run.trial128.stdout: linux-modules-extra-5.15.0-170-generic 2026-02-17T11:20:36.552 INFO:teuthology.orchestra.run.trial128.stdout:The following packages will be upgraded: 2026-02-17T11:20:36.552 INFO:teuthology.orchestra.run.trial128.stdout: linux-generic linux-headers-generic linux-image-generic 2026-02-17T11:20:36.970 INFO:teuthology.orchestra.run.trial128.stdout:3 upgraded, 5 newly installed, 0 to remove and 51 not upgraded. 2026-02-17T11:20:36.970 INFO:teuthology.orchestra.run.trial128.stdout:Need to get 113 MB of archives. 2026-02-17T11:20:36.971 INFO:teuthology.orchestra.run.trial128.stdout:After this operation, 585 MB of additional disk space will be used. 2026-02-17T11:20:36.971 INFO:teuthology.orchestra.run.trial128.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-170-generic amd64 5.15.0-170.180 [22.7 MB] 2026-02-17T11:20:38.307 INFO:teuthology.orchestra.run.trial128.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-170-generic amd64 5.15.0-170.180 [11.6 MB] 2026-02-17T11:20:38.436 INFO:teuthology.orchestra.run.trial128.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-170-generic amd64 5.15.0-170.180 [64.0 MB] 2026-02-17T11:20:38.691 INFO:teuthology.orchestra.run.trial128.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.170.159 [1654 B] 2026-02-17T11:20:38.701 INFO:teuthology.orchestra.run.trial128.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.170.159 [2474 B] 2026-02-17T11:20:38.710 INFO:teuthology.orchestra.run.trial128.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-170 all 5.15.0-170.180 [12.4 MB] 2026-02-17T11:20:38.767 INFO:teuthology.orchestra.run.trial128.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-170-generic amd64 5.15.0-170.180 [2828 kB] 2026-02-17T11:20:38.787 INFO:teuthology.orchestra.run.trial128.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.170.159 [2316 B] 2026-02-17T11:20:38.987 INFO:teuthology.orchestra.run.trial128.stdout:Fetched 113 MB in 2s (50.8 MB/s) 2026-02-17T11:20:39.008 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package linux-modules-5.15.0-170-generic. 2026-02-17T11:20:39.171 INFO:teuthology.orchestra.run.trial128.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 131954 files and directories currently installed.) 2026-02-17T11:20:39.172 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../0-linux-modules-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-17T11:20:39.173 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking linux-modules-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T11:20:39.517 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package linux-image-5.15.0-170-generic. 2026-02-17T11:20:39.523 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../1-linux-image-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-17T11:20:39.542 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T11:20:39.583 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-170-generic. 2026-02-17T11:20:39.589 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-17T11:20:39.590 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking linux-modules-extra-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T11:20:40.691 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../3-linux-generic_5.15.0.170.159_amd64.deb ... 2026-02-17T11:20:40.692 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking linux-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-17T11:20:40.714 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.170.159_amd64.deb ... 2026-02-17T11:20:40.714 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking linux-image-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-17T11:20:40.731 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package linux-headers-5.15.0-170. 2026-02-17T11:20:40.738 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../5-linux-headers-5.15.0-170_5.15.0-170.180_all.deb ... 2026-02-17T11:20:40.738 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking linux-headers-5.15.0-170 (5.15.0-170.180) ... 2026-02-17T11:20:42.110 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package linux-headers-5.15.0-170-generic. 2026-02-17T11:20:42.121 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../6-linux-headers-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-17T11:20:42.122 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking linux-headers-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T11:20:42.720 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.170.159_amd64.deb ... 2026-02-17T11:20:42.721 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking linux-headers-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-17T11:20:42.747 INFO:teuthology.orchestra.run.trial128.stdout:Setting up linux-headers-5.15.0-170 (5.15.0-170.180) ... 2026-02-17T11:20:42.748 INFO:teuthology.orchestra.run.trial128.stdout:Setting up linux-headers-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T11:20:42.750 INFO:teuthology.orchestra.run.trial128.stdout:Setting up linux-headers-generic (5.15.0.170.159) ... 2026-02-17T11:20:42.751 INFO:teuthology.orchestra.run.trial128.stdout:Setting up linux-modules-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T11:20:43.514 INFO:teuthology.orchestra.run.trial128.stdout:Setting up linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T11:20:44.332 INFO:teuthology.orchestra.run.trial128.stdout:I: /boot/vmlinuz is now a symlink to vmlinuz-5.15.0-170-generic 2026-02-17T11:20:44.332 INFO:teuthology.orchestra.run.trial128.stdout:I: /boot/initrd.img is now a symlink to initrd.img-5.15.0-170-generic 2026-02-17T11:20:44.338 INFO:teuthology.orchestra.run.trial128.stdout:Setting up linux-modules-extra-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T11:20:45.125 INFO:teuthology.orchestra.run.trial128.stdout:Setting up linux-image-generic (5.15.0.170.159) ... 2026-02-17T11:20:45.126 INFO:teuthology.orchestra.run.trial128.stdout:Setting up linux-generic (5.15.0.170.159) ... 2026-02-17T11:20:45.127 INFO:teuthology.orchestra.run.trial128.stdout:Processing triggers for linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-17T11:20:45.132 INFO:teuthology.orchestra.run.trial128.stdout:/etc/kernel/postinst.d/initramfs-tools: 2026-02-17T11:20:45.133 INFO:teuthology.orchestra.run.trial128.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-170-generic 2026-02-17T11:20:52.186 INFO:teuthology.orchestra.run.trial128.stdout:W: Possible missing firmware /lib/firmware/ast_dp501_fw.bin for module ast 2026-02-17T11:20:58.835 INFO:teuthology.orchestra.run.trial128.stdout:/etc/kernel/postinst.d/zz-update-grub: 2026-02-17T11:20:58.835 INFO:teuthology.orchestra.run.trial128.stdout:Sourcing file `/etc/default/grub' 2026-02-17T11:20:58.854 INFO:teuthology.orchestra.run.trial128.stdout:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-17T11:20:58.855 INFO:teuthology.orchestra.run.trial128.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-17T11:20:58.857 INFO:teuthology.orchestra.run.trial128.stdout:Generating grub configuration file ... 2026-02-17T11:20:58.935 INFO:teuthology.orchestra.run.trial128.stdout:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-17T11:20:58.942 INFO:teuthology.orchestra.run.trial128.stdout:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-17T11:20:59.111 INFO:teuthology.orchestra.run.trial128.stdout:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-17T11:20:59.116 INFO:teuthology.orchestra.run.trial128.stdout:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-17T11:20:59.187 INFO:teuthology.orchestra.run.trial128.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-17T11:20:59.187 INFO:teuthology.orchestra.run.trial128.stdout:Systems on them will not be added to the GRUB boot configuration. 2026-02-17T11:20:59.187 INFO:teuthology.orchestra.run.trial128.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-17T11:20:59.189 INFO:teuthology.orchestra.run.trial128.stdout:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T11:20:59.198 INFO:teuthology.orchestra.run.trial128.stdout:done 2026-02-17T11:20:59.606 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T11:20:59.607 INFO:teuthology.orchestra.run.trial128.stdout:Pending kernel upgrade! 2026-02-17T11:20:59.607 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T11:20:59.607 INFO:teuthology.orchestra.run.trial128.stdout:Running kernel version: 2026-02-17T11:20:59.607 INFO:teuthology.orchestra.run.trial128.stdout: 5.15.0-164-generic 2026-02-17T11:20:59.607 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T11:20:59.607 INFO:teuthology.orchestra.run.trial128.stdout:Diagnostics: 2026-02-17T11:20:59.607 INFO:teuthology.orchestra.run.trial128.stdout: The currently running kernel version is not the expected kernel version 5.15.0-170-generic. 2026-02-17T11:20:59.607 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T11:20:59.607 INFO:teuthology.orchestra.run.trial128.stdout:Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. [Return] 2026-02-17T11:20:59.608 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T11:20:59.608 INFO:teuthology.orchestra.run.trial128.stdout:The processor microcode seems to be up-to-date. 2026-02-17T11:20:59.608 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T11:20:59.608 INFO:teuthology.orchestra.run.trial128.stdout:No services need to be restarted. 2026-02-17T11:20:59.608 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T11:20:59.608 INFO:teuthology.orchestra.run.trial128.stdout:No containers need to be restarted. 2026-02-17T11:20:59.608 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T11:20:59.608 INFO:teuthology.orchestra.run.trial128.stdout:No user sessions are running outdated binaries. 2026-02-17T11:20:59.608 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T11:20:59.608 INFO:teuthology.orchestra.run.trial128.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-17T11:21:00.272 DEBUG:teuthology.orchestra.run.trial128:> dpkg -s linux-image-generic 2026-02-17T11:21:00.281 INFO:teuthology.orchestra.run.trial128.stdout:Package: linux-image-generic 2026-02-17T11:21:00.282 INFO:teuthology.orchestra.run.trial128.stdout:Status: install ok installed 2026-02-17T11:21:00.282 INFO:teuthology.orchestra.run.trial128.stdout:Priority: optional 2026-02-17T11:21:00.282 INFO:teuthology.orchestra.run.trial128.stdout:Section: kernel 2026-02-17T11:21:00.282 INFO:teuthology.orchestra.run.trial128.stdout:Installed-Size: 21 2026-02-17T11:21:00.282 INFO:teuthology.orchestra.run.trial128.stdout:Maintainer: Ubuntu Kernel Team 2026-02-17T11:21:00.282 INFO:teuthology.orchestra.run.trial128.stdout:Architecture: amd64 2026-02-17T11:21:00.282 INFO:teuthology.orchestra.run.trial128.stdout:Source: linux-meta 2026-02-17T11:21:00.282 INFO:teuthology.orchestra.run.trial128.stdout:Version: 5.15.0.170.159 2026-02-17T11:21:00.283 INFO:teuthology.orchestra.run.trial128.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-17T11:21:00.283 INFO:teuthology.orchestra.run.trial128.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-17T11:21:00.283 INFO:teuthology.orchestra.run.trial128.stdout:Recommends: thermald 2026-02-17T11:21:00.283 INFO:teuthology.orchestra.run.trial128.stdout:Description: Generic Linux kernel image 2026-02-17T11:21:00.283 INFO:teuthology.orchestra.run.trial128.stdout: This package will always depend on the latest generic kernel image 2026-02-17T11:21:00.283 INFO:teuthology.orchestra.run.trial128.stdout: available. 2026-02-17T11:21:00.284 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-164-generic Expected: 5.15.0-170-generic 2026-02-17T11:21:00.284 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-17T11:21:00.284 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-17T11:21:00.285 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-17T11:21:00.285 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial128.front.sepia.ceph.com, path=None, version=distro) 2026-02-17T11:21:00.285 DEBUG:teuthology.orchestra.run.trial128:> sudo apt-get clean 2026-02-17T11:21:00.366 DEBUG:teuthology.orchestra.run.trial128:> sudo apt-get update 2026-02-17T11:21:00.812 INFO:teuthology.orchestra.run.trial128.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-17T11:21:00.879 INFO:teuthology.orchestra.run.trial128.stdout:Hit:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-17T11:21:00.952 INFO:teuthology.orchestra.run.trial128.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-17T11:21:01.024 INFO:teuthology.orchestra.run.trial128.stdout:Hit:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-17T11:21:01.663 INFO:teuthology.orchestra.run.trial128.stdout:Reading package lists... 2026-02-17T11:21:01.677 DEBUG:teuthology.orchestra.run.trial128:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-17T11:21:01.710 INFO:teuthology.orchestra.run.trial128.stdout:Reading package lists... 2026-02-17T11:21:01.822 INFO:teuthology.orchestra.run.trial128.stdout:Building dependency tree... 2026-02-17T11:21:01.822 INFO:teuthology.orchestra.run.trial128.stdout:Reading state information... 2026-02-17T11:21:01.906 INFO:teuthology.orchestra.run.trial128.stdout:linux-image-generic is already the newest version (5.15.0.170.159). 2026-02-17T11:21:01.906 INFO:teuthology.orchestra.run.trial128.stdout:linux-image-generic set to manually installed. 2026-02-17T11:21:01.906 INFO:teuthology.orchestra.run.trial128.stdout:The following packages were automatically installed and are no longer required: 2026-02-17T11:21:01.906 INFO:teuthology.orchestra.run.trial128.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-17T11:21:01.906 INFO:teuthology.orchestra.run.trial128.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-17T11:21:01.906 INFO:teuthology.orchestra.run.trial128.stdout: sg3-utils-udev 2026-02-17T11:21:01.907 INFO:teuthology.orchestra.run.trial128.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-17T11:21:01.925 INFO:teuthology.orchestra.run.trial128.stdout:0 upgraded, 0 newly installed, 0 to remove and 51 not upgraded. 2026-02-17T11:21:01.928 DEBUG:teuthology.orchestra.run.trial128:> dpkg -s linux-image-generic 2026-02-17T11:21:01.978 INFO:teuthology.orchestra.run.trial128.stdout:Package: linux-image-generic 2026-02-17T11:21:01.978 INFO:teuthology.orchestra.run.trial128.stdout:Status: install ok installed 2026-02-17T11:21:01.978 INFO:teuthology.orchestra.run.trial128.stdout:Priority: optional 2026-02-17T11:21:01.979 INFO:teuthology.orchestra.run.trial128.stdout:Section: kernel 2026-02-17T11:21:01.979 INFO:teuthology.orchestra.run.trial128.stdout:Installed-Size: 21 2026-02-17T11:21:01.979 INFO:teuthology.orchestra.run.trial128.stdout:Maintainer: Ubuntu Kernel Team 2026-02-17T11:21:01.979 INFO:teuthology.orchestra.run.trial128.stdout:Architecture: amd64 2026-02-17T11:21:01.979 INFO:teuthology.orchestra.run.trial128.stdout:Source: linux-meta 2026-02-17T11:21:01.979 INFO:teuthology.orchestra.run.trial128.stdout:Version: 5.15.0.170.159 2026-02-17T11:21:01.979 INFO:teuthology.orchestra.run.trial128.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-17T11:21:01.979 INFO:teuthology.orchestra.run.trial128.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-17T11:21:01.979 INFO:teuthology.orchestra.run.trial128.stdout:Recommends: thermald 2026-02-17T11:21:01.979 INFO:teuthology.orchestra.run.trial128.stdout:Description: Generic Linux kernel image 2026-02-17T11:21:01.979 INFO:teuthology.orchestra.run.trial128.stdout: This package will always depend on the latest generic kernel image 2026-02-17T11:21:01.980 INFO:teuthology.orchestra.run.trial128.stdout: available. 2026-02-17T11:21:01.980 DEBUG:teuthology.orchestra.run.trial128:> mktemp 2026-02-17T11:21:02.025 INFO:teuthology.orchestra.run.trial128.stdout:/tmp/tmp.tUXeZYU6fy 2026-02-17T11:21:02.026 DEBUG:teuthology.orchestra.run.trial128:> sudo cp /boot/grub/grub.cfg /tmp/tmp.tUXeZYU6fy 2026-02-17T11:21:02.078 DEBUG:teuthology.orchestra.run.trial128:> sudo chmod 0666 /tmp/tmp.tUXeZYU6fy 2026-02-17T11:21:02.174 DEBUG:teuthology.orchestra.remote:trial128:/tmp/tmp.tUXeZYU6fy is 7KB 2026-02-17T11:21:02.180 DEBUG:teuthology.orchestra.run.trial128:> rm -fr /tmp/tmp.tUXeZYU6fy 2026-02-17T11:21:02.184 DEBUG:teuthology.orchestra.run.trial128:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2026-02-17T11:21:02.237 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T11:21:02.237 DEBUG:teuthology.orchestra.run.trial128:> sudo dd of=/etc/grub.d/01_ceph_kernel 2026-02-17T11:21:02.237 DEBUG:teuthology.orchestra.run.trial128:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2026-02-17T11:21:02.291 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-170-generic 2026-02-17T11:21:02.291 DEBUG:teuthology.orchestra.run.trial128:> sudo update-grub 2026-02-17T11:21:02.529 INFO:teuthology.orchestra.run.trial128.stderr:Sourcing file `/etc/default/grub' 2026-02-17T11:21:02.549 INFO:teuthology.orchestra.run.trial128.stderr:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-17T11:21:02.550 INFO:teuthology.orchestra.run.trial128.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-17T11:21:02.552 INFO:teuthology.orchestra.run.trial128.stderr:Generating grub configuration file ... 2026-02-17T11:21:02.632 INFO:teuthology.orchestra.run.trial128.stderr:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-17T11:21:02.638 INFO:teuthology.orchestra.run.trial128.stderr:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-17T11:21:02.811 INFO:teuthology.orchestra.run.trial128.stderr:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-17T11:21:02.816 INFO:teuthology.orchestra.run.trial128.stderr:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-17T11:21:02.886 INFO:teuthology.orchestra.run.trial128.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-17T11:21:02.886 INFO:teuthology.orchestra.run.trial128.stderr:Systems on them will not be added to the GRUB boot configuration. 2026-02-17T11:21:02.886 INFO:teuthology.orchestra.run.trial128.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-17T11:21:02.888 INFO:teuthology.orchestra.run.trial128.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T11:21:02.895 INFO:teuthology.orchestra.run.trial128.stderr:done 2026-02-17T11:21:02.897 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial128 2026-02-17T11:21:02.897 DEBUG:teuthology.orchestra.run.trial128:> sudo bash -c ' 2026-02-17T11:21:02.897 DEBUG:teuthology.orchestra.run.trial128:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-17T11:21:02.897 DEBUG:teuthology.orchestra.run.trial128:> ' 2026-02-17T11:21:32.899 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-17T11:21:32.900 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial128.front.sepia.ceph.com' 2026-02-17T11:21:32.901 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial128.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T11:21:47.200 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.128 2026-02-17T11:21:56.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial128.front.sepia.ceph.com' 2026-02-17T11:21:56.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial128.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T11:22:32.352 DEBUG:teuthology.orchestra.run.trial128:> true 2026-02-17T11:22:32.763 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial128.front.sepia.ceph.com' 2026-02-17T11:22:32.763 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-170-generic"... 2026-02-17T11:22:32.764 DEBUG:teuthology.orchestra.run.trial128:> uname -r 2026-02-17T11:22:32.808 INFO:teuthology.orchestra.run.trial128.stdout:5.15.0-170-generic 2026-02-17T11:22:32.808 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-170-generic vs 5.15.0-170-generic 2026-02-17T11:22:32.808 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-17T11:22:32.808 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2026-02-17T11:22:33.809 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-17T11:22:33.810 DEBUG:teuthology.orchestra.run.trial128:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-17T11:22:33.823 INFO:teuthology.orchestra.run.trial128.stdout:ttyS1 2026-02-17T11:22:33.858 DEBUG:teuthology.parallel:result is None 2026-02-17T11:22:33.858 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-17T11:22:33.867 INFO:teuthology.task.internal:Creating test directory... 2026-02-17T11:22:33.868 DEBUG:teuthology.orchestra.run.trial128:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-17T11:22:33.904 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-17T11:22:33.917 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-17T11:22:33.930 INFO:teuthology.task.internal:Creating archive directory... 2026-02-17T11:22:33.930 DEBUG:teuthology.orchestra.run.trial128:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-17T11:22:33.955 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-17T11:22:33.966 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-17T11:22:33.967 DEBUG:teuthology.orchestra.run.trial128:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-17T11:22:34.001 INFO:teuthology.orchestra.run.trial128.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:22:34.007 INFO:teuthology.orchestra.run.trial128.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:22:34.009 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-17T11:22:34.023 INFO:teuthology.task.internal:Configuring sudo... 2026-02-17T11:22:34.023 DEBUG:teuthology.orchestra.run.trial128:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-17T11:22:34.060 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-17T11:22:34.076 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-17T11:22:34.077 DEBUG:teuthology.orchestra.run.trial128:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-17T11:22:34.108 DEBUG:teuthology.orchestra.run.trial128:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T11:22:34.152 DEBUG:teuthology.orchestra.run.trial128:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T11:22:34.196 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T11:22:34.196 DEBUG:teuthology.orchestra.run.trial128:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-17T11:22:34.245 DEBUG:teuthology.orchestra.run.trial128:> sudo service rsyslog restart 2026-02-17T11:22:34.305 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-17T11:22:34.317 INFO:teuthology.task.internal:Starting timer... 2026-02-17T11:22:34.317 INFO:teuthology.run_tasks:Running task pcp... 2026-02-17T11:22:34.340 INFO:teuthology.run_tasks:Running task selinux... 2026-02-17T11:22:34.357 DEBUG:teuthology.task.selinux:Excluding trial128: OS 'ubuntu' does not support SELinux 2026-02-17T11:22:34.357 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-17T11:22:34.357 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-17T11:22:34.357 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-17T11:22:34.357 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-17T11:22:34.373 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-17T11:22:34.376 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-17T11:22:34.478 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-17T11:22:34.490 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-17T11:22:34.491 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial128.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-17T11:24:59.134 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial128.front.sepia.ceph.com')] 2026-02-17T11:24:59.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial128.front.sepia.ceph.com' 2026-02-17T11:24:59.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial128.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T11:24:59.243 DEBUG:teuthology.orchestra.run.trial128:> true 2026-02-17T11:24:59.499 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial128.front.sepia.ceph.com' 2026-02-17T11:24:59.499 INFO:teuthology.run_tasks:Running task clock... 2026-02-17T11:24:59.512 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-17T11:24:59.512 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-17T11:24:59.513 DEBUG:teuthology.orchestra.run.trial128:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T11:24:59.560 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-17T11:24:59.560 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: Command line: ntpd -gq 2026-02-17T11:24:59.560 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: ---------------------------------------------------- 2026-02-17T11:24:59.560 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: ntp-4 is maintained by Network Time Foundation, 2026-02-17T11:24:59.560 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-17T11:24:59.560 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: corporation. Support and training for ntp-4 are 2026-02-17T11:24:59.560 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: available at https://www.nwtime.org/support 2026-02-17T11:24:59.560 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: ---------------------------------------------------- 2026-02-17T11:24:59.561 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: proto: precision = 0.020 usec (-25) 2026-02-17T11:24:59.561 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: basedate set to 2022-02-04 2026-02-17T11:24:59.561 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: gps base set to 2022-02-06 (week 2196) 2026-02-17T11:24:59.561 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-17T11:24:59.561 INFO:teuthology.orchestra.run.trial128.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-17T11:24:59.561 INFO:teuthology.orchestra.run.trial128.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-17T11:24:59.562 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: restrict ::: KOD does nothing without LIMITED. 2026-02-17T11:24:59.562 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: Listen and drop on 0 v6wildcard [::]:123 2026-02-17T11:24:59.562 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-17T11:24:59.562 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: Listen normally on 2 lo 127.0.0.1:123 2026-02-17T11:24:59.562 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: Listen normally on 3 enp1s0f0 10.20.193.128:123 2026-02-17T11:24:59.562 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: Listen normally on 4 lo [::1]:123 2026-02-17T11:24:59.562 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:6222%2]:123 2026-02-17T11:24:59.562 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:24:59 ntpd[8785]: Listening on routing socket on fd #22 for interface updates 2026-02-17T11:25:00.560 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:00 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:00.561 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:00 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:00.591 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:00 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:00.591 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:00 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:00.591 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:00 ntpd[8785]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T11:25:00.591 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:00 ntpd[8785]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T11:25:02.560 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:02 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:02.560 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:02 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:02.560 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:02 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:02.560 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:02 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:02.563 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:02 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:02.563 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:02 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:02.563 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:02 ntpd[8785]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T11:25:02.563 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:02 ntpd[8785]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T11:25:02.590 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:02 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:02.590 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:02 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:02.590 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:02 ntpd[8785]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T11:25:02.590 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:02 ntpd[8785]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T11:25:03.560 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:03 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:03.560 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:03 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:03.649 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:03 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:03.649 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:03 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:03.649 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:03 ntpd[8785]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T11:25:03.649 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:03 ntpd[8785]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T11:25:04.560 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:04.560 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:04.560 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:04.560 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:04.560 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:04.560 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:04.563 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:04.563 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:04.563 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T11:25:04.563 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T11:25:04.591 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:04.591 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:04.591 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T11:25:04.591 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T11:25:04.625 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:04.626 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:04.626 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T11:25:04.626 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:04 ntpd[8785]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T11:25:05.560 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:05 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:05.561 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:05 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:05.649 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:05 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:05.649 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:05 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:05.649 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:05 ntpd[8785]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T11:25:05.649 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:05 ntpd[8785]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T11:25:06.561 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:06 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:06.561 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:06 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:06.561 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:06 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:06.561 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:06 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:06.561 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:06 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:06.561 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:06 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:06.563 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:06 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:06.564 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:06 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:06.564 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:06 ntpd[8785]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T11:25:06.564 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:06 ntpd[8785]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T11:25:06.591 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:06 ntpd[8785]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-17T11:25:06.591 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:06 ntpd[8785]: can't open /var/log/ntpstats/rawstats.20260217: Permission denied 2026-02-17T11:25:06.591 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:06 ntpd[8785]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-17T11:25:06.591 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:06 ntpd[8785]: can't open /var/log/ntpstats/peerstats.20260217: Permission denied 2026-02-17T11:25:06.591 INFO:teuthology.orchestra.run.trial128.stdout:17 Feb 11:25:06 ntpd[8785]: ntpd: time set -0.295703 s 2026-02-17T11:25:06.591 INFO:teuthology.orchestra.run.trial128.stdout:ntpd: time set -0.295703s 2026-02-17T11:25:06.592 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:06 ntpd[8785]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-17T11:25:06.592 INFO:teuthology.orchestra.run.trial128.stderr:17 Feb 11:25:06 ntpd[8785]: can't open /var/log/ntpstats/loopstats.20260217: Permission denied 2026-02-17T11:25:06.616 INFO:teuthology.orchestra.run.trial128.stdout:No association ID's returned 2026-02-17T11:25:06.617 INFO:teuthology.run_tasks:Running task install... 2026-02-17T11:25:06.632 DEBUG:teuthology.task.install:project ceph 2026-02-17T11:25:06.632 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3c96dfb3f50b3d02598a82af7104764bfdcd3919'}} 2026-02-17T11:25:06.632 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '3c96dfb3f50b3d02598a82af7104764bfdcd3919'} 2026-02-17T11:25:06.633 INFO:teuthology.task.install:Using flavor: default 2026-02-17T11:25:06.641 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-17T11:25:06.642 INFO:teuthology.task.install:extra packages: [] 2026-02-17T11:25:06.642 DEBUG:teuthology.orchestra.run.trial128:> sudo apt-key list | grep Ceph 2026-02-17T11:25:06.709 INFO:teuthology.orchestra.run.trial128.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-17T11:25:06.737 INFO:teuthology.orchestra.run.trial128.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-17T11:25:06.738 INFO:teuthology.orchestra.run.trial128.stdout:uid [ unknown] Ceph.com (release key) 2026-02-17T11:25:06.739 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-17T11:25:06.740 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3c96dfb3f50b3d02598a82af7104764bfdcd3919 2026-02-17T11:25:06.920 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default/ 2026-02-17T11:25:06.920 INFO:teuthology.task.install.deb:Package version is 19.2.3-675-g3c96dfb3-1jammy 2026-02-17T11:25:07.083 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T11:25:07.083 DEBUG:teuthology.orchestra.run.trial128:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-17T11:25:07.092 DEBUG:teuthology.orchestra.run.trial128:> sudo apt-get update 2026-02-17T11:25:07.253 INFO:teuthology.orchestra.run.trial128.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy InRelease 2026-02-17T11:25:07.277 INFO:teuthology.orchestra.run.trial128.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy Release [6978 B] 2026-02-17T11:25:07.284 INFO:teuthology.orchestra.run.trial128.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-17T11:25:07.288 INFO:teuthology.orchestra.run.trial128.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-17T11:25:07.289 INFO:teuthology.orchestra.run.trial128.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-17T11:25:07.299 INFO:teuthology.orchestra.run.trial128.stdout:Hit:6 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-17T11:25:07.308 INFO:teuthology.orchestra.run.trial128.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-17T11:25:07.331 INFO:teuthology.orchestra.run.trial128.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2026-02-17T11:25:07.627 INFO:teuthology.orchestra.run.trial128.stdout:Fetched 25.1 kB in 0s (69.4 kB/s) 2026-02-17T11:25:08.138 INFO:teuthology.orchestra.run.trial128.stdout:Reading package lists... 2026-02-17T11:25:08.151 DEBUG:teuthology.orchestra.run.trial128:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.2.3-675-g3c96dfb3-1jammy cephadm=19.2.3-675-g3c96dfb3-1jammy ceph-mds=19.2.3-675-g3c96dfb3-1jammy ceph-mgr=19.2.3-675-g3c96dfb3-1jammy ceph-common=19.2.3-675-g3c96dfb3-1jammy ceph-fuse=19.2.3-675-g3c96dfb3-1jammy ceph-test=19.2.3-675-g3c96dfb3-1jammy ceph-volume=19.2.3-675-g3c96dfb3-1jammy radosgw=19.2.3-675-g3c96dfb3-1jammy python3-rados=19.2.3-675-g3c96dfb3-1jammy python3-rgw=19.2.3-675-g3c96dfb3-1jammy python3-cephfs=19.2.3-675-g3c96dfb3-1jammy python3-rbd=19.2.3-675-g3c96dfb3-1jammy libcephfs2=19.2.3-675-g3c96dfb3-1jammy libcephfs-dev=19.2.3-675-g3c96dfb3-1jammy librados2=19.2.3-675-g3c96dfb3-1jammy librbd1=19.2.3-675-g3c96dfb3-1jammy rbd-fuse=19.2.3-675-g3c96dfb3-1jammy 2026-02-17T11:25:08.184 INFO:teuthology.orchestra.run.trial128.stdout:Reading package lists... 2026-02-17T11:25:08.292 INFO:teuthology.orchestra.run.trial128.stdout:Building dependency tree... 2026-02-17T11:25:08.292 INFO:teuthology.orchestra.run.trial128.stdout:Reading state information... 2026-02-17T11:25:08.381 INFO:teuthology.orchestra.run.trial128.stdout:The following packages were automatically installed and are no longer required: 2026-02-17T11:25:08.381 INFO:teuthology.orchestra.run.trial128.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-17T11:25:08.382 INFO:teuthology.orchestra.run.trial128.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-17T11:25:08.382 INFO:teuthology.orchestra.run.trial128.stdout: sg3-utils-udev 2026-02-17T11:25:08.382 INFO:teuthology.orchestra.run.trial128.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-17T11:25:08.382 INFO:teuthology.orchestra.run.trial128.stdout:The following additional packages will be installed: 2026-02-17T11:25:08.382 INFO:teuthology.orchestra.run.trial128.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-17T11:25:08.382 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2026-02-17T11:25:08.382 INFO:teuthology.orchestra.run.trial128.stdout: libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 2026-02-17T11:25:08.382 INFO:teuthology.orchestra.run.trial128.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-17T11:25:08.382 INFO:teuthology.orchestra.run.trial128.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2026-02-17T11:25:08.382 INFO:teuthology.orchestra.run.trial128.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2026-02-17T11:25:08.383 INFO:teuthology.orchestra.run.trial128.stdout: pkg-config python-asyncssh-doc python-pastedeploy-tpl python3-asyncssh 2026-02-17T11:25:08.383 INFO:teuthology.orchestra.run.trial128.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cheroot 2026-02-17T11:25:08.383 INFO:teuthology.orchestra.run.trial128.stdout: python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-17T11:25:08.383 INFO:teuthology.orchestra.run.trial128.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-17T11:25:08.383 INFO:teuthology.orchestra.run.trial128.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-logutils 2026-02-17T11:25:08.383 INFO:teuthology.orchestra.run.trial128.stdout: python3-mako python3-natsort python3-paste python3-pastedeploy 2026-02-17T11:25:08.383 INFO:teuthology.orchestra.run.trial128.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2026-02-17T11:25:08.383 INFO:teuthology.orchestra.run.trial128.stdout: python3-prettytable python3-psutil python3-py python3-pygments 2026-02-17T11:25:08.383 INFO:teuthology.orchestra.run.trial128.stdout: python3-pyinotify python3-pytest python3-repoze.lru 2026-02-17T11:25:08.383 INFO:teuthology.orchestra.run.trial128.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2026-02-17T11:25:08.384 INFO:teuthology.orchestra.run.trial128.stdout: python3-simplejson python3-singledispatch python3-sklearn 2026-02-17T11:25:08.384 INFO:teuthology.orchestra.run.trial128.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2026-02-17T11:25:08.384 INFO:teuthology.orchestra.run.trial128.stdout: python3-toml python3-waitress python3-wcwidth python3-webob 2026-02-17T11:25:08.384 INFO:teuthology.orchestra.run.trial128.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2026-02-17T11:25:08.384 INFO:teuthology.orchestra.run.trial128.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2026-02-17T11:25:08.384 INFO:teuthology.orchestra.run.trial128.stdout:Suggested packages: 2026-02-17T11:25:08.384 INFO:teuthology.orchestra.run.trial128.stdout: python3-influxdb readline-doc python3-beaker python-mako-doc 2026-02-17T11:25:08.384 INFO:teuthology.orchestra.run.trial128.stdout: python-natsort-doc httpd-wsgi libapache2-mod-python libapache2-mod-scgi 2026-02-17T11:25:08.384 INFO:teuthology.orchestra.run.trial128.stdout: libjs-mochikit python-pecan-doc python-psutil-doc subversion 2026-02-17T11:25:08.384 INFO:teuthology.orchestra.run.trial128.stdout: python-pygments-doc ttf-bitstream-vera python-pyinotify-doc python3-dap 2026-02-17T11:25:08.384 INFO:teuthology.orchestra.run.trial128.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2026-02-17T11:25:08.384 INFO:teuthology.orchestra.run.trial128.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2026-02-17T11:25:08.385 INFO:teuthology.orchestra.run.trial128.stdout:Recommended packages: 2026-02-17T11:25:08.385 INFO:teuthology.orchestra.run.trial128.stdout: btrfs-tools 2026-02-17T11:25:08.416 INFO:teuthology.orchestra.run.trial128.stdout:The following NEW packages will be installed: 2026-02-17T11:25:08.416 INFO:teuthology.orchestra.run.trial128.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-17T11:25:08.416 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-17T11:25:08.416 INFO:teuthology.orchestra.run.trial128.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2026-02-17T11:25:08.416 INFO:teuthology.orchestra.run.trial128.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 2026-02-17T11:25:08.416 INFO:teuthology.orchestra.run.trial128.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2026-02-17T11:25:08.417 INFO:teuthology.orchestra.run.trial128.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2026-02-17T11:25:08.417 INFO:teuthology.orchestra.run.trial128.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2026-02-17T11:25:08.417 INFO:teuthology.orchestra.run.trial128.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2026-02-17T11:25:08.417 INFO:teuthology.orchestra.run.trial128.stdout: python-pastedeploy-tpl python3-asyncssh python3-cachetools 2026-02-17T11:25:08.417 INFO:teuthology.orchestra.run.trial128.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cheroot 2026-02-17T11:25:08.417 INFO:teuthology.orchestra.run.trial128.stdout: python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-17T11:25:08.417 INFO:teuthology.orchestra.run.trial128.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-17T11:25:08.417 INFO:teuthology.orchestra.run.trial128.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-logutils 2026-02-17T11:25:08.418 INFO:teuthology.orchestra.run.trial128.stdout: python3-mako python3-natsort python3-paste python3-pastedeploy 2026-02-17T11:25:08.418 INFO:teuthology.orchestra.run.trial128.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2026-02-17T11:25:08.418 INFO:teuthology.orchestra.run.trial128.stdout: python3-prettytable python3-psutil python3-py python3-pygments 2026-02-17T11:25:08.418 INFO:teuthology.orchestra.run.trial128.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2026-02-17T11:25:08.418 INFO:teuthology.orchestra.run.trial128.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2026-02-17T11:25:08.418 INFO:teuthology.orchestra.run.trial128.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2026-02-17T11:25:08.418 INFO:teuthology.orchestra.run.trial128.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2026-02-17T11:25:08.418 INFO:teuthology.orchestra.run.trial128.stdout: python3-threadpoolctl python3-toml python3-waitress python3-wcwidth 2026-02-17T11:25:08.418 INFO:teuthology.orchestra.run.trial128.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2026-02-17T11:25:08.418 INFO:teuthology.orchestra.run.trial128.stdout: python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse socat unzip 2026-02-17T11:25:08.418 INFO:teuthology.orchestra.run.trial128.stdout: xmlstarlet zip 2026-02-17T11:25:08.567 INFO:teuthology.orchestra.run.trial128.stdout:0 upgraded, 108 newly installed, 0 to remove and 51 not upgraded. 2026-02-17T11:25:08.567 INFO:teuthology.orchestra.run.trial128.stdout:Need to get 178 MB of archives. 2026-02-17T11:25:08.567 INFO:teuthology.orchestra.run.trial128.stdout:After this operation, 809 MB of additional disk space will be used. 2026-02-17T11:25:08.567 INFO:teuthology.orchestra.run.trial128.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.2.3-675-g3c96dfb3-1jammy [3597 kB] 2026-02-17T11:25:08.686 INFO:teuthology.orchestra.run.trial128.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.2.3-675-g3c96dfb3-1jammy [3257 kB] 2026-02-17T11:25:08.769 INFO:teuthology.orchestra.run.trial128.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-17T11:25:08.792 INFO:teuthology.orchestra.run.trial128.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.2.3-675-g3c96dfb3-1jammy [979 kB] 2026-02-17T11:25:08.855 INFO:teuthology.orchestra.run.trial128.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.2.3-675-g3c96dfb3-1jammy [356 kB] 2026-02-17T11:25:08.889 INFO:teuthology.orchestra.run.trial128.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.2.3-675-g3c96dfb3-1jammy [32.9 kB] 2026-02-17T11:25:08.912 INFO:teuthology.orchestra.run.trial128.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.2.3-675-g3c96dfb3-1jammy [184 kB] 2026-02-17T11:25:08.931 INFO:teuthology.orchestra.run.trial128.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.2.3-675-g3c96dfb3-1jammy [70.1 kB] 2026-02-17T11:25:08.965 INFO:teuthology.orchestra.run.trial128.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.2.3-675-g3c96dfb3-1jammy [336 kB] 2026-02-17T11:25:09.009 INFO:teuthology.orchestra.run.trial128.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.2.3-675-g3c96dfb3-1jammy [6935 kB] 2026-02-17T11:25:09.106 INFO:teuthology.orchestra.run.trial128.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-17T11:25:09.117 INFO:teuthology.orchestra.run.trial128.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-17T11:25:09.121 INFO:teuthology.orchestra.run.trial128.stdout:Get:13 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.2.3-675-g3c96dfb3-1jammy [112 kB] 2026-02-17T11:25:09.163 INFO:teuthology.orchestra.run.trial128.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.2.3-675-g3c96dfb3-1jammy [470 kB] 2026-02-17T11:25:09.186 INFO:teuthology.orchestra.run.trial128.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-17T11:25:09.206 INFO:teuthology.orchestra.run.trial128.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.2.3-675-g3c96dfb3-1jammy [26.5 MB] 2026-02-17T11:25:09.387 INFO:teuthology.orchestra.run.trial128.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-17T11:25:09.397 INFO:teuthology.orchestra.run.trial128.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-17T11:25:09.423 INFO:teuthology.orchestra.run.trial128.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-17T11:25:09.431 INFO:teuthology.orchestra.run.trial128.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-17T11:25:09.433 INFO:teuthology.orchestra.run.trial128.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-17T11:25:09.434 INFO:teuthology.orchestra.run.trial128.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-17T11:25:09.435 INFO:teuthology.orchestra.run.trial128.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-17T11:25:09.452 INFO:teuthology.orchestra.run.trial128.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2026-02-17T11:25:09.455 INFO:teuthology.orchestra.run.trial128.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2026-02-17T11:25:09.459 INFO:teuthology.orchestra.run.trial128.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2026-02-17T11:25:09.528 INFO:teuthology.orchestra.run.trial128.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5034 B] 2026-02-17T11:25:09.528 INFO:teuthology.orchestra.run.trial128.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2026-02-17T11:25:09.529 INFO:teuthology.orchestra.run.trial128.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2026-02-17T11:25:09.531 INFO:teuthology.orchestra.run.trial128.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2026-02-17T11:25:09.533 INFO:teuthology.orchestra.run.trial128.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-17T11:25:09.533 INFO:teuthology.orchestra.run.trial128.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-17T11:25:09.534 INFO:teuthology.orchestra.run.trial128.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-17T11:25:09.534 INFO:teuthology.orchestra.run.trial128.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-17T11:25:09.605 INFO:teuthology.orchestra.run.trial128.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-17T11:25:09.605 INFO:teuthology.orchestra.run.trial128.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-17T11:25:09.605 INFO:teuthology.orchestra.run.trial128.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-17T11:25:09.606 INFO:teuthology.orchestra.run.trial128.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-17T11:25:09.606 INFO:teuthology.orchestra.run.trial128.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-17T11:25:09.676 INFO:teuthology.orchestra.run.trial128.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-17T11:25:09.678 INFO:teuthology.orchestra.run.trial128.stdout:Get:41 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-17T11:25:09.679 INFO:teuthology.orchestra.run.trial128.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2026-02-17T11:25:09.679 INFO:teuthology.orchestra.run.trial128.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2026-02-17T11:25:09.680 INFO:teuthology.orchestra.run.trial128.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2026-02-17T11:25:09.735 INFO:teuthology.orchestra.run.trial128.stdout:Get:45 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.2.3-675-g3c96dfb3-1jammy [5178 kB] 2026-02-17T11:25:09.748 INFO:teuthology.orchestra.run.trial128.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7320 B] 2026-02-17T11:25:09.748 INFO:teuthology.orchestra.run.trial128.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-17T11:25:09.749 INFO:teuthology.orchestra.run.trial128.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-waitress all 1.4.4-1.1ubuntu1.1 [47.0 kB] 2026-02-17T11:25:09.749 INFO:teuthology.orchestra.run.trial128.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2026-02-17T11:25:09.750 INFO:teuthology.orchestra.run.trial128.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2026-02-17T11:25:09.820 INFO:teuthology.orchestra.run.trial128.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4892 B] 2026-02-17T11:25:09.820 INFO:teuthology.orchestra.run.trial128.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2026-02-17T11:25:09.820 INFO:teuthology.orchestra.run.trial128.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2026-02-17T11:25:09.820 INFO:teuthology.orchestra.run.trial128.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2026-02-17T11:25:09.821 INFO:teuthology.orchestra.run.trial128.stdout:Get:55 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.3 [181 kB] 2026-02-17T11:25:09.892 INFO:teuthology.orchestra.run.trial128.stdout:Get:56 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-17T11:25:09.893 INFO:teuthology.orchestra.run.trial128.stdout:Get:57 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-17T11:25:09.895 INFO:teuthology.orchestra.run.trial128.stdout:Get:58 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-17T11:25:09.895 INFO:teuthology.orchestra.run.trial128.stdout:Get:59 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-17T11:25:09.896 INFO:teuthology.orchestra.run.trial128.stdout:Get:60 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-17T11:25:09.955 INFO:teuthology.orchestra.run.trial128.stdout:Get:61 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.2.3-675-g3c96dfb3-1jammy [248 kB] 2026-02-17T11:25:09.964 INFO:teuthology.orchestra.run.trial128.stdout:Get:62 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-17T11:25:09.966 INFO:teuthology.orchestra.run.trial128.stdout:Get:63 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-17T11:25:09.966 INFO:teuthology.orchestra.run.trial128.stdout:Get:64 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-17T11:25:09.978 INFO:teuthology.orchestra.run.trial128.stdout:Get:65 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.2.3-675-g3c96dfb3-1jammy [124 kB] 2026-02-17T11:25:09.986 INFO:teuthology.orchestra.run.trial128.stdout:Get:66 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-17T11:25:09.986 INFO:teuthology.orchestra.run.trial128.stdout:Get:67 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-17T11:25:10.033 INFO:teuthology.orchestra.run.trial128.stdout:Get:68 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.2.3-675-g3c96dfb3-1jammy [1081 kB] 2026-02-17T11:25:10.035 INFO:teuthology.orchestra.run.trial128.stdout:Get:69 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-17T11:25:10.036 INFO:teuthology.orchestra.run.trial128.stdout:Get:70 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-17T11:25:10.036 INFO:teuthology.orchestra.run.trial128.stdout:Get:71 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-17T11:25:10.037 INFO:teuthology.orchestra.run.trial128.stdout:Get:72 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-17T11:25:10.040 INFO:teuthology.orchestra.run.trial128.stdout:Get:73 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-17T11:25:10.099 INFO:teuthology.orchestra.run.trial128.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.2.3-675-g3c96dfb3-1jammy [6239 kB] 2026-02-17T11:25:10.108 INFO:teuthology.orchestra.run.trial128.stdout:Get:75 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-17T11:25:10.109 INFO:teuthology.orchestra.run.trial128.stdout:Get:76 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-17T11:25:10.110 INFO:teuthology.orchestra.run.trial128.stdout:Get:77 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-17T11:25:10.113 INFO:teuthology.orchestra.run.trial128.stdout:Get:78 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-17T11:25:10.116 INFO:teuthology.orchestra.run.trial128.stdout:Get:79 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2026-02-17T11:25:10.180 INFO:teuthology.orchestra.run.trial128.stdout:Get:80 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2026-02-17T11:25:10.180 INFO:teuthology.orchestra.run.trial128.stdout:Get:81 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-17T11:25:10.185 INFO:teuthology.orchestra.run.trial128.stdout:Get:82 https://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2026-02-17T11:25:10.185 INFO:teuthology.orchestra.run.trial128.stdout:Get:83 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-17T11:25:10.188 INFO:teuthology.orchestra.run.trial128.stdout:Get:84 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-17T11:25:10.254 INFO:teuthology.orchestra.run.trial128.stdout:Get:85 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2026-02-17T11:25:10.255 INFO:teuthology.orchestra.run.trial128.stdout:Get:86 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-17T11:25:10.256 INFO:teuthology.orchestra.run.trial128.stdout:Get:87 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-17T11:25:10.257 INFO:teuthology.orchestra.run.trial128.stdout:Get:88 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-17T11:25:10.259 INFO:teuthology.orchestra.run.trial128.stdout:Get:89 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-17T11:25:10.326 INFO:teuthology.orchestra.run.trial128.stdout:Get:90 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2026-02-17T11:25:10.326 INFO:teuthology.orchestra.run.trial128.stdout:Get:91 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-17T11:25:10.326 INFO:teuthology.orchestra.run.trial128.stdout:Get:92 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-17T11:25:10.328 INFO:teuthology.orchestra.run.trial128.stdout:Get:93 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-17T11:25:10.328 INFO:teuthology.orchestra.run.trial128.stdout:Get:94 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-17T11:25:10.358 INFO:teuthology.orchestra.run.trial128.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.2.3-675-g3c96dfb3-1jammy [23.0 MB] 2026-02-17T11:25:11.023 INFO:teuthology.orchestra.run.trial128.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.2.3-675-g3c96dfb3-1jammy [14.2 kB] 2026-02-17T11:25:11.024 INFO:teuthology.orchestra.run.trial128.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.2.3-675-g3c96dfb3-1jammy [1173 kB] 2026-02-17T11:25:11.069 INFO:teuthology.orchestra.run.trial128.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.2.3-675-g3c96dfb3-1jammy [2503 kB] 2026-02-17T11:25:11.165 INFO:teuthology.orchestra.run.trial128.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.2.3-675-g3c96dfb3-1jammy [798 kB] 2026-02-17T11:25:11.714 INFO:teuthology.orchestra.run.trial128.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.2.3-675-g3c96dfb3-1jammy [157 kB] 2026-02-17T11:25:11.735 INFO:teuthology.orchestra.run.trial128.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.2.3-675-g3c96dfb3-1jammy [2396 kB] 2026-02-17T11:25:11.840 INFO:teuthology.orchestra.run.trial128.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.2.3-675-g3c96dfb3-1jammy [8625 kB] 2026-02-17T11:25:12.086 INFO:teuthology.orchestra.run.trial128.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.2.3-675-g3c96dfb3-1jammy [14.3 kB] 2026-02-17T11:25:12.130 INFO:teuthology.orchestra.run.trial128.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.2.3-675-g3c96dfb3-1jammy [52.1 MB] 2026-02-17T11:25:13.510 INFO:teuthology.orchestra.run.trial128.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.2.3-675-g3c96dfb3-1jammy [135 kB] 2026-02-17T11:25:13.531 INFO:teuthology.orchestra.run.trial128.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.2.3-675-g3c96dfb3-1jammy [41.0 kB] 2026-02-17T11:25:13.555 INFO:teuthology.orchestra.run.trial128.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.2.3-675-g3c96dfb3-1jammy [13.7 MB] 2026-02-17T11:25:13.857 INFO:teuthology.orchestra.run.trial128.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-bharath20-testing-2026-02-13-0906-squid/3c96dfb3f50b3d02598a82af7104764bfdcd3919/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.2.3-675-g3c96dfb3-1jammy [92.1 kB] 2026-02-17T11:25:14.200 INFO:teuthology.orchestra.run.trial128.stdout:Fetched 178 MB in 5s (32.5 MB/s) 2026-02-17T11:25:14.217 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-17T11:25:14.370 INFO:teuthology.orchestra.run.trial128.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167438 files and directories currently installed.) 2026-02-17T11:25:14.372 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-17T11:25:14.372 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-17T11:25:14.393 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-17T11:25:14.406 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-17T11:25:14.406 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-17T11:25:14.421 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-17T11:25:14.433 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-17T11:25:14.434 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-17T11:25:14.452 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-17T11:25:14.460 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-17T11:25:14.466 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T11:25:14.497 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-17T11:25:14.505 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-17T11:25:14.506 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T11:25:14.522 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-17T11:25:14.531 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-17T11:25:14.531 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T11:25:14.551 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-17T11:25:14.559 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-17T11:25:14.559 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-17T11:25:14.576 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package librados2. 2026-02-17T11:25:14.584 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../007-librados2_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:14.584 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking librados2 (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:14.628 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package libnbd0. 2026-02-17T11:25:14.636 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2026-02-17T11:25:14.636 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-17T11:25:14.650 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package librbd1. 2026-02-17T11:25:14.658 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../009-librbd1_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:14.658 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking librbd1 (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:14.699 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package libcephfs2. 2026-02-17T11:25:14.707 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../010-libcephfs2_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:14.708 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking libcephfs2 (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:14.728 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-rados. 2026-02-17T11:25:14.736 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../011-python3-rados_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:14.737 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-rados (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:14.755 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-17T11:25:14.763 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.2.3-675-g3c96dfb3-1jammy_all.deb ... 2026-02-17T11:25:14.763 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-ceph-argparse (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:14.778 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-cephfs. 2026-02-17T11:25:14.786 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../013-python3-cephfs_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:14.786 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-cephfs (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:14.802 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-17T11:25:14.811 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../014-python3-ceph-common_19.2.3-675-g3c96dfb3-1jammy_all.deb ... 2026-02-17T11:25:14.811 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-ceph-common (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:14.828 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-17T11:25:14.837 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-17T11:25:14.837 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-17T11:25:14.853 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-prettytable. 2026-02-17T11:25:14.861 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2026-02-17T11:25:14.862 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-17T11:25:14.876 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-rbd. 2026-02-17T11:25:14.884 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../017-python3-rbd_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:14.885 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-rbd (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:14.902 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-17T11:25:14.910 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-17T11:25:14.910 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-17T11:25:14.928 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package libreadline-dev:amd64. 2026-02-17T11:25:14.936 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2026-02-17T11:25:14.936 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2026-02-17T11:25:14.954 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2026-02-17T11:25:14.962 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2026-02-17T11:25:14.962 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2026-02-17T11:25:14.980 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package lua5.1. 2026-02-17T11:25:14.988 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2026-02-17T11:25:14.989 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2026-02-17T11:25:15.010 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package lua-any. 2026-02-17T11:25:15.018 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2026-02-17T11:25:15.018 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking lua-any (27ubuntu1) ... 2026-02-17T11:25:15.033 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package zip. 2026-02-17T11:25:15.041 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2026-02-17T11:25:15.041 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking zip (3.0-12build2) ... 2026-02-17T11:25:15.057 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package unzip. 2026-02-17T11:25:15.065 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2026-02-17T11:25:15.066 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2026-02-17T11:25:15.083 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package luarocks. 2026-02-17T11:25:15.091 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2026-02-17T11:25:15.091 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2026-02-17T11:25:15.130 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package librgw2. 2026-02-17T11:25:15.138 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../026-librgw2_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:15.139 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking librgw2 (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:15.209 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-rgw. 2026-02-17T11:25:15.218 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../027-python3-rgw_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:15.218 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-rgw (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:15.233 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-17T11:25:15.241 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../028-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-17T11:25:15.241 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-17T11:25:15.256 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package libradosstriper1. 2026-02-17T11:25:15.264 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../029-libradosstriper1_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:15.265 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking libradosstriper1 (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:15.283 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package ceph-common. 2026-02-17T11:25:15.291 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../030-ceph-common_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:15.292 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking ceph-common (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:15.495 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package ceph-base. 2026-02-17T11:25:15.503 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../031-ceph-base_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:15.509 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking ceph-base (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:15.573 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-17T11:25:15.581 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-17T11:25:15.582 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-17T11:25:15.597 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-cheroot. 2026-02-17T11:25:15.605 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-17T11:25:15.605 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-17T11:25:15.623 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-17T11:25:15.631 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-17T11:25:15.631 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-17T11:25:15.647 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-17T11:25:15.655 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-17T11:25:15.655 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-17T11:25:15.669 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-17T11:25:15.677 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-17T11:25:15.678 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-17T11:25:15.692 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-tempora. 2026-02-17T11:25:15.700 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2026-02-17T11:25:15.701 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-17T11:25:15.716 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-portend. 2026-02-17T11:25:15.725 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2026-02-17T11:25:15.726 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-17T11:25:15.740 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-17T11:25:15.748 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-17T11:25:15.749 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-17T11:25:15.763 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-17T11:25:15.771 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-17T11:25:15.772 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-17T11:25:15.795 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-natsort. 2026-02-17T11:25:15.803 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2026-02-17T11:25:15.803 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-17T11:25:15.820 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-logutils. 2026-02-17T11:25:15.828 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2026-02-17T11:25:15.828 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-logutils (0.3.3-8) ... 2026-02-17T11:25:15.843 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-mako. 2026-02-17T11:25:15.851 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2026-02-17T11:25:15.852 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2026-02-17T11:25:15.869 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-simplegeneric. 2026-02-17T11:25:15.877 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2026-02-17T11:25:15.878 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2026-02-17T11:25:15.892 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-singledispatch. 2026-02-17T11:25:15.900 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2026-02-17T11:25:15.901 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2026-02-17T11:25:15.914 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-webob. 2026-02-17T11:25:15.922 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-17T11:25:15.923 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-17T11:25:15.937 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-waitress. 2026-02-17T11:25:15.946 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1.1_all.deb ... 2026-02-17T11:25:15.948 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1.1) ... 2026-02-17T11:25:15.963 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-tempita. 2026-02-17T11:25:15.972 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2026-02-17T11:25:15.972 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2026-02-17T11:25:15.987 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-paste. 2026-02-17T11:25:15.996 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2026-02-17T11:25:15.996 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2026-02-17T11:25:16.024 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2026-02-17T11:25:16.032 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2026-02-17T11:25:16.033 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2026-02-17T11:25:16.046 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-pastedeploy. 2026-02-17T11:25:16.055 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2026-02-17T11:25:16.055 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2026-02-17T11:25:16.070 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-webtest. 2026-02-17T11:25:16.079 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2026-02-17T11:25:16.079 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-webtest (2.0.35-1) ... 2026-02-17T11:25:16.095 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-pecan. 2026-02-17T11:25:16.103 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2026-02-17T11:25:16.103 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2026-02-17T11:25:16.129 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-werkzeug. 2026-02-17T11:25:16.137 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.3_all.deb ... 2026-02-17T11:25:16.137 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.3) ... 2026-02-17T11:25:16.157 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-17T11:25:16.165 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_19.2.3-675-g3c96dfb3-1jammy_all.deb ... 2026-02-17T11:25:16.165 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking ceph-mgr-modules-core (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:16.192 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-17T11:25:16.201 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:16.201 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking libsqlite3-mod-ceph (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:16.216 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package ceph-mgr. 2026-02-17T11:25:16.224 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../057-ceph-mgr_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:16.225 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking ceph-mgr (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:16.247 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package ceph-mon. 2026-02-17T11:25:16.255 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../058-ceph-mon_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:16.256 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking ceph-mon (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:16.313 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-17T11:25:16.321 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../059-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-17T11:25:16.322 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-17T11:25:16.338 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package ceph-osd. 2026-02-17T11:25:16.346 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../060-ceph-osd_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:16.346 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking ceph-osd (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:16.504 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package ceph. 2026-02-17T11:25:16.512 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../061-ceph_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:16.513 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking ceph (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:16.526 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package ceph-fuse. 2026-02-17T11:25:16.534 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../062-ceph-fuse_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:16.534 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking ceph-fuse (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:16.557 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package ceph-mds. 2026-02-17T11:25:16.566 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../063-ceph-mds_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:16.566 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking ceph-mds (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:16.600 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package cephadm. 2026-02-17T11:25:16.608 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../064-cephadm_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:16.609 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking cephadm (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:16.625 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-17T11:25:16.633 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-17T11:25:16.633 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-17T11:25:16.655 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-17T11:25:16.663 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../066-ceph-mgr-cephadm_19.2.3-675-g3c96dfb3-1jammy_all.deb ... 2026-02-17T11:25:16.663 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking ceph-mgr-cephadm (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:16.684 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-17T11:25:16.692 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../067-python3-repoze.lru_0.7-2_all.deb ... 2026-02-17T11:25:16.692 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-17T11:25:16.707 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-routes. 2026-02-17T11:25:16.716 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../068-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-17T11:25:16.716 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-17T11:25:16.736 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-17T11:25:16.744 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../069-ceph-mgr-dashboard_19.2.3-675-g3c96dfb3-1jammy_all.deb ... 2026-02-17T11:25:16.745 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking ceph-mgr-dashboard (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:16.945 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-17T11:25:16.954 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-17T11:25:16.954 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-17T11:25:16.994 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-joblib. 2026-02-17T11:25:17.002 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../071-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-17T11:25:17.003 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-17T11:25:17.029 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-17T11:25:17.037 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../072-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-17T11:25:17.038 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-17T11:25:17.053 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-sklearn. 2026-02-17T11:25:17.061 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../073-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-17T11:25:17.062 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-17T11:25:17.140 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-17T11:25:17.149 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../074-ceph-mgr-diskprediction-local_19.2.3-675-g3c96dfb3-1jammy_all.deb ... 2026-02-17T11:25:17.149 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking ceph-mgr-diskprediction-local (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:17.315 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-cachetools. 2026-02-17T11:25:17.324 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../075-python3-cachetools_5.0.0-1_all.deb ... 2026-02-17T11:25:17.325 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-17T11:25:17.340 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-rsa. 2026-02-17T11:25:17.348 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../076-python3-rsa_4.8-1_all.deb ... 2026-02-17T11:25:17.349 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-17T11:25:17.366 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-google-auth. 2026-02-17T11:25:17.375 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-3_all.deb ... 2026-02-17T11:25:17.375 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-17T11:25:17.392 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-17T11:25:17.401 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-17T11:25:17.401 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-17T11:25:17.417 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-websocket. 2026-02-17T11:25:17.426 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../079-python3-websocket_1.2.3-1_all.deb ... 2026-02-17T11:25:17.427 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-17T11:25:17.444 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-17T11:25:17.453 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../080-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-17T11:25:17.453 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-17T11:25:17.548 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-17T11:25:17.557 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_19.2.3-675-g3c96dfb3-1jammy_all.deb ... 2026-02-17T11:25:17.558 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking ceph-mgr-k8sevents (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:17.573 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-17T11:25:17.581 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../082-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-17T11:25:17.588 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-17T11:25:17.604 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-17T11:25:17.613 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../083-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-17T11:25:17.613 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-17T11:25:17.628 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package jq. 2026-02-17T11:25:17.636 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../084-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-17T11:25:17.637 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-17T11:25:17.651 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package socat. 2026-02-17T11:25:17.660 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../085-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-17T11:25:17.661 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-17T11:25:17.680 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package xmlstarlet. 2026-02-17T11:25:17.688 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-17T11:25:17.688 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-17T11:25:17.725 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package ceph-test. 2026-02-17T11:25:17.734 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../087-ceph-test_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:17.734 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking ceph-test (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:18.199 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package ceph-volume. 2026-02-17T11:25:18.208 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../088-ceph-volume_19.2.3-675-g3c96dfb3-1jammy_all.deb ... 2026-02-17T11:25:18.208 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking ceph-volume (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:18.231 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-17T11:25:18.240 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../089-libcephfs-dev_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:18.241 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking libcephfs-dev (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:18.256 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package lua-socket:amd64. 2026-02-17T11:25:18.265 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../090-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2026-02-17T11:25:18.265 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2026-02-17T11:25:18.286 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package lua-sec:amd64. 2026-02-17T11:25:18.295 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../091-lua-sec_1.0.2-1_amd64.deb ... 2026-02-17T11:25:18.295 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2026-02-17T11:25:18.312 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package nvme-cli. 2026-02-17T11:25:18.321 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../092-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-17T11:25:18.321 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-17T11:25:18.352 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package pkg-config. 2026-02-17T11:25:18.361 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../093-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2026-02-17T11:25:18.361 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2026-02-17T11:25:18.377 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-17T11:25:18.385 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../094-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-17T11:25:18.386 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-17T11:25:18.413 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-17T11:25:18.422 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../095-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-17T11:25:18.422 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-17T11:25:18.436 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-pastescript. 2026-02-17T11:25:18.445 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../096-python3-pastescript_2.0.2-4_all.deb ... 2026-02-17T11:25:18.445 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2026-02-17T11:25:18.464 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-pluggy. 2026-02-17T11:25:18.472 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../097-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-17T11:25:18.473 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-17T11:25:18.489 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-psutil. 2026-02-17T11:25:18.497 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../098-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-17T11:25:18.498 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-17T11:25:18.516 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-py. 2026-02-17T11:25:18.525 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../099-python3-py_1.10.0-1_all.deb ... 2026-02-17T11:25:18.525 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-17T11:25:18.545 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-pygments. 2026-02-17T11:25:18.553 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../100-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-17T11:25:18.554 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-17T11:25:18.596 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-pyinotify. 2026-02-17T11:25:18.605 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../101-python3-pyinotify_0.9.6-1.3_all.deb ... 2026-02-17T11:25:18.605 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2026-02-17T11:25:18.620 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-toml. 2026-02-17T11:25:18.629 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../102-python3-toml_0.10.2-1_all.deb ... 2026-02-17T11:25:18.629 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-17T11:25:18.644 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-pytest. 2026-02-17T11:25:18.653 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../103-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-17T11:25:18.653 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-17T11:25:18.674 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package python3-simplejson. 2026-02-17T11:25:18.682 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../104-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-17T11:25:18.683 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-17T11:25:18.700 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-17T11:25:18.709 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../105-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-17T11:25:18.709 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-17T11:25:18.775 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package radosgw. 2026-02-17T11:25:18.784 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../106-radosgw_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:18.784 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking radosgw (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:18.899 INFO:teuthology.orchestra.run.trial128.stdout:Selecting previously unselected package rbd-fuse. 2026-02-17T11:25:18.907 INFO:teuthology.orchestra.run.trial128.stdout:Preparing to unpack .../107-rbd-fuse_19.2.3-675-g3c96dfb3-1jammy_amd64.deb ... 2026-02-17T11:25:18.908 INFO:teuthology.orchestra.run.trial128.stdout:Unpacking rbd-fuse (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:18.936 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-17T11:25:19.003 INFO:teuthology.orchestra.run.trial128.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-17T11:25:19.004 INFO:teuthology.orchestra.run.trial128.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-17T11:25:19.076 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-17T11:25:19.350 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-17T11:25:19.714 INFO:teuthology.orchestra.run.trial128.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-17T11:25:19.725 INFO:teuthology.orchestra.run.trial128.stdout:Setting up cephadm (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:19.769 INFO:teuthology.orchestra.run.trial128.stdout:Adding system user cephadm....done 2026-02-17T11:25:19.778 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1.1) ... 2026-02-17T11:25:19.853 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-17T11:25:19.923 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-17T11:25:19.924 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-17T11:25:19.996 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-17T11:25:20.071 INFO:teuthology.orchestra.run.trial128.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-17T11:25:20.072 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-17T11:25:20.169 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-17T11:25:20.293 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-17T11:25:20.367 INFO:teuthology.orchestra.run.trial128.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2026-02-17T11:25:20.379 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2026-02-17T11:25:20.461 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-17T11:25:20.534 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-ceph-argparse (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:20.602 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-17T11:25:20.603 INFO:teuthology.orchestra.run.trial128.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-17T11:25:20.604 INFO:teuthology.orchestra.run.trial128.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2026-02-17T11:25:20.605 INFO:teuthology.orchestra.run.trial128.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2026-02-17T11:25:20.605 INFO:teuthology.orchestra.run.trial128.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-17T11:25:20.606 INFO:teuthology.orchestra.run.trial128.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2026-02-17T11:25:20.622 INFO:teuthology.orchestra.run.trial128.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2026-02-17T11:25:20.624 INFO:teuthology.orchestra.run.trial128.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2026-02-17T11:25:20.625 INFO:teuthology.orchestra.run.trial128.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-17T11:25:20.626 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-17T11:25:20.751 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-17T11:25:20.830 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-17T11:25:20.908 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-17T11:25:20.998 INFO:teuthology.orchestra.run.trial128.stdout:Setting up zip (3.0-12build2) ... 2026-02-17T11:25:20.999 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-17T11:25:21.264 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2026-02-17T11:25:21.341 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2026-02-17T11:25:21.342 INFO:teuthology.orchestra.run.trial128.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-17T11:25:21.343 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-17T11:25:21.439 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-17T11:25:21.566 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2026-02-17T11:25:21.698 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-17T11:25:21.781 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.3) ... 2026-02-17T11:25:21.889 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-17T11:25:21.961 INFO:teuthology.orchestra.run.trial128.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-17T11:25:21.962 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-ceph-common (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:22.046 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-17T11:25:22.566 INFO:teuthology.orchestra.run.trial128.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2026-02-17T11:25:22.588 INFO:teuthology.orchestra.run.trial128.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T11:25:22.592 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-17T11:25:22.668 INFO:teuthology.orchestra.run.trial128.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-17T11:25:22.669 INFO:teuthology.orchestra.run.trial128.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-17T11:25:22.670 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-17T11:25:22.747 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-17T11:25:22.821 INFO:teuthology.orchestra.run.trial128.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T11:25:22.821 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-17T11:25:22.898 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2026-02-17T11:25:22.972 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-logutils (0.3.3-8) ... 2026-02-17T11:25:23.048 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-17T11:25:23.123 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2026-02-17T11:25:23.195 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-17T11:25:23.267 INFO:teuthology.orchestra.run.trial128.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-17T11:25:23.268 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-17T11:25:23.352 INFO:teuthology.orchestra.run.trial128.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-17T11:25:23.353 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-17T11:25:23.431 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2026-02-17T11:25:23.514 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-17T11:25:23.600 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-17T11:25:23.674 INFO:teuthology.orchestra.run.trial128.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2026-02-17T11:25:23.675 INFO:teuthology.orchestra.run.trial128.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2026-02-17T11:25:23.676 INFO:teuthology.orchestra.run.trial128.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-17T11:25:23.677 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-17T11:25:23.814 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2026-02-17T11:25:23.892 INFO:teuthology.orchestra.run.trial128.stdout:Setting up lua-any (27ubuntu1) ... 2026-02-17T11:25:23.893 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-17T11:25:23.968 INFO:teuthology.orchestra.run.trial128.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-17T11:25:23.969 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-17T11:25:24.054 INFO:teuthology.orchestra.run.trial128.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-17T11:25:24.054 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-webtest (2.0.35-1) ... 2026-02-17T11:25:24.136 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-17T11:25:24.275 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-pastescript (2.0.2-4) ... 2026-02-17T11:25:24.365 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2026-02-17T11:25:24.478 INFO:teuthology.orchestra.run.trial128.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-17T11:25:24.479 INFO:teuthology.orchestra.run.trial128.stdout:Setting up librados2 (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:24.480 INFO:teuthology.orchestra.run.trial128.stdout:Setting up libsqlite3-mod-ceph (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:24.481 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-17T11:25:25.016 INFO:teuthology.orchestra.run.trial128.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2026-02-17T11:25:25.026 INFO:teuthology.orchestra.run.trial128.stdout:Setting up libcephfs2 (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:25.027 INFO:teuthology.orchestra.run.trial128.stdout:Setting up libradosstriper1 (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:25.028 INFO:teuthology.orchestra.run.trial128.stdout:Setting up librbd1 (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:25.029 INFO:teuthology.orchestra.run.trial128.stdout:Setting up ceph-mgr-modules-core (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:25.030 INFO:teuthology.orchestra.run.trial128.stdout:Setting up ceph-fuse (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:25.099 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-17T11:25:25.100 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-17T11:25:25.501 INFO:teuthology.orchestra.run.trial128.stdout:Setting up libcephfs-dev (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:25.501 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-rados (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:25.502 INFO:teuthology.orchestra.run.trial128.stdout:Setting up librgw2 (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:25.503 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-rbd (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:25.504 INFO:teuthology.orchestra.run.trial128.stdout:Setting up rbd-fuse (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:25.505 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-rgw (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:25.506 INFO:teuthology.orchestra.run.trial128.stdout:Setting up python3-cephfs (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:25.507 INFO:teuthology.orchestra.run.trial128.stdout:Setting up ceph-common (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:25.535 INFO:teuthology.orchestra.run.trial128.stdout:Adding group ceph....done 2026-02-17T11:25:25.571 INFO:teuthology.orchestra.run.trial128.stdout:Adding system user ceph....done 2026-02-17T11:25:25.580 INFO:teuthology.orchestra.run.trial128.stdout:Setting system user ceph properties....done 2026-02-17T11:25:25.587 INFO:teuthology.orchestra.run.trial128.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-17T11:25:25.665 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-17T11:25:25.918 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-17T11:25:26.290 INFO:teuthology.orchestra.run.trial128.stdout:Setting up ceph-test (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:26.291 INFO:teuthology.orchestra.run.trial128.stdout:Setting up radosgw (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:26.538 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-17T11:25:26.538 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-17T11:25:26.890 INFO:teuthology.orchestra.run.trial128.stdout:Setting up ceph-base (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:27.007 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-17T11:25:27.354 INFO:teuthology.orchestra.run.trial128.stdout:Setting up ceph-mds (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:27.415 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-17T11:25:27.415 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-17T11:25:27.784 INFO:teuthology.orchestra.run.trial128.stdout:Setting up ceph-mgr (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:27.862 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-17T11:25:27.862 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-17T11:25:28.225 INFO:teuthology.orchestra.run.trial128.stdout:Setting up ceph-osd (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:28.309 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-17T11:25:28.310 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-17T11:25:28.681 INFO:teuthology.orchestra.run.trial128.stdout:Setting up ceph-mgr-k8sevents (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:28.682 INFO:teuthology.orchestra.run.trial128.stdout:Setting up ceph-mgr-diskprediction-local (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:28.692 INFO:teuthology.orchestra.run.trial128.stdout:Setting up ceph-mon (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:28.765 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-17T11:25:28.765 INFO:teuthology.orchestra.run.trial128.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-17T11:25:29.157 INFO:teuthology.orchestra.run.trial128.stdout:Setting up ceph-mgr-cephadm (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:29.169 INFO:teuthology.orchestra.run.trial128.stdout:Setting up ceph (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:29.170 INFO:teuthology.orchestra.run.trial128.stdout:Setting up ceph-mgr-dashboard (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:29.180 INFO:teuthology.orchestra.run.trial128.stdout:Setting up ceph-volume (19.2.3-675-g3c96dfb3-1jammy) ... 2026-02-17T11:25:29.291 INFO:teuthology.orchestra.run.trial128.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2026-02-17T11:25:29.299 INFO:teuthology.orchestra.run.trial128.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.11) ... 2026-02-17T11:25:29.318 INFO:teuthology.orchestra.run.trial128.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-17T11:25:29.408 INFO:teuthology.orchestra.run.trial128.stdout:Processing triggers for install-info (6.8-4build1) ... 2026-02-17T11:25:29.911 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T11:25:29.911 INFO:teuthology.orchestra.run.trial128.stdout:Running kernel seems to be up-to-date. 2026-02-17T11:25:29.911 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T11:25:29.911 INFO:teuthology.orchestra.run.trial128.stdout:The processor microcode seems to be up-to-date. 2026-02-17T11:25:29.911 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T11:25:29.911 INFO:teuthology.orchestra.run.trial128.stdout:No services need to be restarted. 2026-02-17T11:25:29.911 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T11:25:29.911 INFO:teuthology.orchestra.run.trial128.stdout:No containers need to be restarted. 2026-02-17T11:25:29.911 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T11:25:29.911 INFO:teuthology.orchestra.run.trial128.stdout:No user sessions are running outdated binaries. 2026-02-17T11:25:29.911 INFO:teuthology.orchestra.run.trial128.stdout: 2026-02-17T11:25:29.911 INFO:teuthology.orchestra.run.trial128.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-17T11:25:30.566 INFO:teuthology.orchestra.run.trial128.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-17T11:25:30.574 DEBUG:teuthology.parallel:result is None 2026-02-17T11:25:30.574 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3c96dfb3f50b3d02598a82af7104764bfdcd3919 2026-02-17T11:25:30.722 DEBUG:teuthology.orchestra.run.trial128:> dpkg-query -W -f '${Version}' ceph 2026-02-17T11:25:30.731 INFO:teuthology.orchestra.run.trial128.stdout:19.2.3-675-g3c96dfb3-1jammy 2026-02-17T11:25:30.732 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-675-g3c96dfb3-1jammy 2026-02-17T11:25:30.732 INFO:teuthology.task.install:The correct ceph version 19.2.3-675-g3c96dfb3-1jammy is installed. 2026-02-17T11:25:30.733 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-17T11:25:30.733 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T11:25:30.733 DEBUG:teuthology.orchestra.run.trial128:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-17T11:25:30.784 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-17T11:25:30.784 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T11:25:30.784 DEBUG:teuthology.orchestra.run.trial128:> sudo dd of=/usr/bin/daemon-helper 2026-02-17T11:25:30.835 DEBUG:teuthology.orchestra.run.trial128:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-17T11:25:30.883 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-17T11:25:30.883 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T11:25:30.883 DEBUG:teuthology.orchestra.run.trial128:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-17T11:25:30.969 DEBUG:teuthology.orchestra.run.trial128:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-17T11:25:31.018 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-17T11:25:31.018 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T11:25:31.018 DEBUG:teuthology.orchestra.run.trial128:> sudo dd of=/usr/bin/stdin-killer 2026-02-17T11:25:31.069 DEBUG:teuthology.orchestra.run.trial128:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-17T11:25:31.118 INFO:teuthology.run_tasks:Running task workunit... 2026-02-17T11:25:31.133 INFO:tasks.workunit:Pulling workunits from ref 3c96dfb3f50b3d02598a82af7104764bfdcd3919 2026-02-17T11:25:31.133 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-17T11:25:31.133 INFO:tasks.workunit:timeout=3h 2026-02-17T11:25:31.133 INFO:tasks.workunit:cleanup=True 2026-02-17T11:25:31.133 DEBUG:teuthology.orchestra.run.trial128:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-17T11:25:31.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T11:25:31.162 INFO:teuthology.orchestra.run.trial128.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-17T11:25:31.162 DEBUG:teuthology.orchestra.run.trial128:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-17T11:25:31.206 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-17T11:25:31.206 DEBUG:teuthology.orchestra.run.trial128:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-17T11:25:31.250 DEBUG:teuthology.orchestra.run.trial128:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3c96dfb3f50b3d02598a82af7104764bfdcd3919 2026-02-17T11:25:31.302 INFO:tasks.workunit.client.0.trial128.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-17T11:25:53.229 INFO:tasks.workunit.client.0.trial128.stderr:Note: switching to '3c96dfb3f50b3d02598a82af7104764bfdcd3919'. 2026-02-17T11:25:53.229 INFO:tasks.workunit.client.0.trial128.stderr: 2026-02-17T11:25:53.229 INFO:tasks.workunit.client.0.trial128.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-17T11:25:53.229 INFO:tasks.workunit.client.0.trial128.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-17T11:25:53.230 INFO:tasks.workunit.client.0.trial128.stderr:state without impacting any branches by switching back to a branch. 2026-02-17T11:25:53.230 INFO:tasks.workunit.client.0.trial128.stderr: 2026-02-17T11:25:53.230 INFO:tasks.workunit.client.0.trial128.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-17T11:25:53.230 INFO:tasks.workunit.client.0.trial128.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-17T11:25:53.230 INFO:tasks.workunit.client.0.trial128.stderr: 2026-02-17T11:25:53.230 INFO:tasks.workunit.client.0.trial128.stderr: git switch -c 2026-02-17T11:25:53.230 INFO:tasks.workunit.client.0.trial128.stderr: 2026-02-17T11:25:53.230 INFO:tasks.workunit.client.0.trial128.stderr:Or undo this operation with: 2026-02-17T11:25:53.230 INFO:tasks.workunit.client.0.trial128.stderr: 2026-02-17T11:25:53.230 INFO:tasks.workunit.client.0.trial128.stderr: git switch - 2026-02-17T11:25:53.230 INFO:tasks.workunit.client.0.trial128.stderr: 2026-02-17T11:25:53.231 INFO:tasks.workunit.client.0.trial128.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-17T11:25:53.231 INFO:tasks.workunit.client.0.trial128.stderr: 2026-02-17T11:25:53.231 INFO:tasks.workunit.client.0.trial128.stderr:HEAD is now at 3c96dfb3f50 Merge branch 'wip-ifed-non-spatial-alloc-squi' of https://github.com/ifed01/ceph into wip-bharath20-testing-2026-02-13-0906-squid 2026-02-17T11:25:53.240 DEBUG:teuthology.orchestra.run.trial128:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-17T11:25:53.289 DEBUG:teuthology.orchestra.run.trial128:> set -ex 2026-02-17T11:25:53.289 DEBUG:teuthology.orchestra.run.trial128:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-17T11:25:53.336 INFO:tasks.workunit:Running workunits matching scrub on client.0... 2026-02-17T11:25:53.337 INFO:tasks.workunit:Running workunit scrub/osd-mapper.sh... 2026-02-17T11:25:53.337 DEBUG:teuthology.orchestra.run.trial128:workunit test scrub/osd-mapper.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3c96dfb3f50b3d02598a82af7104764bfdcd3919 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh 2026-02-17T11:25:53.390 INFO:tasks.workunit.client.0.trial128.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T11:25:53.395 INFO:tasks.workunit.client.0.trial128.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T11:25:53.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T11:25:53.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T11:25:53.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T11:25:53.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T11:25:53.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T11:25:53.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T11:25:53.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T11:25:53.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T11:25:53.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-mapper 2026-02-17T11:25:53.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:11: run: local dir=td/osd-mapper 2026-02-17T11:25:53.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:12: run: shift 2026-02-17T11:25:53.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:14: run: export CEPH_MON=127.0.0.1:7144 2026-02-17T11:25:53.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:14: run: CEPH_MON=127.0.0.1:7144 2026-02-17T11:25:53.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:15: run: export CEPH_ARGS 2026-02-17T11:25:53.397 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:16: run: uuidgen 2026-02-17T11:25:53.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:16: run: CEPH_ARGS+='--fsid=fdb2547e-2a2a-41a0-b169-5dfc682b2ffb --auth-supported=none ' 2026-02-17T11:25:53.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:17: run: CEPH_ARGS+='--mon-host=127.0.0.1:7144 ' 2026-02-17T11:25:53.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:19: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-17T11:25:53.398 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:20: run: set 2026-02-17T11:25:53.398 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:20: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T11:25:53.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:20: run: local funcs=TEST_truncated_sna_record 2026-02-17T11:25:53.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:21: run: for func in $funcs 2026-02-17T11:25:53.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:22: run: setup td/osd-mapper 2026-02-17T11:25:53.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-mapper 2026-02-17T11:25:53.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-mapper 2026-02-17T11:25:53.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-mapper 2026-02-17T11:25:53.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:25:53.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-mapper KILL 2026-02-17T11:25:53.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:25:53.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:25:53.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:25:53.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:25:53.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:25:53.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:25:53.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:25:53.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:25:53.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:25:53.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:25:53.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:25:53.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:25:53.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:25:53.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:25:53.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:25:53.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:25:53.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:25:53.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:25:53.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-mapper 2026-02-17T11:25:53.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:25:53.411 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:25:53.411 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11816 2026-02-17T11:25:53.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.11816 2026-02-17T11:25:53.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:25:53.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:25:53.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-mapper 2026-02-17T11:25:53.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:25:53.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:25:53.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11816 2026-02-17T11:25:53.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.11816 2026-02-17T11:25:53.415 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:25:53.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T11:25:53.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T11:25:53.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:25:53.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-mapper 1' TERM HUP INT 2026-02-17T11:25:53.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:23: run: TEST_truncated_sna_record td/osd-mapper 2026-02-17T11:25:53.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:45: TEST_truncated_sna_record: local dir=td/osd-mapper 2026-02-17T11:25:53.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:46: TEST_truncated_sna_record: cluster_conf=(['osds_num']='3' ['pgs_in_pool']='4' ['pool_name']='test') 2026-02-17T11:25:53.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:46: TEST_truncated_sna_record: local -A cluster_conf 2026-02-17T11:25:53.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:52: TEST_truncated_sna_record: local extr_dbg=3 2026-02-17T11:25:53.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:53: TEST_truncated_sna_record: (( extr_dbg > 1 )) 2026-02-17T11:25:53.418 INFO:tasks.workunit.client.0.trial128.stdout:Dir: td/osd-mapper 2026-02-17T11:25:53.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:53: TEST_truncated_sna_record: echo 'Dir: td/osd-mapper' 2026-02-17T11:25:53.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:54: TEST_truncated_sna_record: standard_scrub_cluster td/osd-mapper cluster_conf 2026-02-17T11:25:53.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:229: standard_scrub_cluster: local dir=td/osd-mapper 2026-02-17T11:25:53.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:230: standard_scrub_cluster: local -n args=cluster_conf 2026-02-17T11:25:53.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:232: standard_scrub_cluster: local OSDS=3 2026-02-17T11:25:53.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:233: standard_scrub_cluster: local pg_num=4 2026-02-17T11:25:53.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:234: standard_scrub_cluster: local poolname=test 2026-02-17T11:25:53.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:235: standard_scrub_cluster: args['pool_name']=test 2026-02-17T11:25:53.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:236: standard_scrub_cluster: local extra_pars= 2026-02-17T11:25:53.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:237: standard_scrub_cluster: local debug_msg=dbg 2026-02-17T11:25:53.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:240: standard_scrub_cluster: local saved_echo_flag=x 2026-02-17T11:25:53.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:241: standard_scrub_cluster: set +x 2026-02-17T11:25:53.712 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 b706816b-c6ef-45c0-a473-57c73761fb52 2026-02-17T11:25:53.810 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:25:53.835 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:25:53.827+0000 7fbc7d43f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:25:53.836 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:25:53.831+0000 7fbc7d43f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:25:53.837 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:25:53.831+0000 7fbc7d43f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:25:53.838 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:25:53.831+0000 7fbc7d43f8c0 -1 bdev(0x55583da5dc00 td/osd-mapper/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:25:53.838 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:25:53.831+0000 7fbc7d43f8c0 -1 bluestore(td/osd-mapper/0) _read_fsid unparsable uuid 2026-02-17T11:25:56.396 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:25:56.584 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:25:56.608 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:25:56.603+0000 7f23dd5328c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:25:56.609 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:25:56.603+0000 7f23dd5328c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:25:56.611 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:25:56.603+0000 7f23dd5328c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:25:56.739 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:25:57.894 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:25:58.085 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:25:58.079+0000 7f23dd5328c0 -1 Falling back to public interface 2026-02-17T11:25:59.048 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:26:00.203 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:26:01.053 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:01.047+0000 7f23dd5328c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:26:01.355 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:26:02.506 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:26:02.659 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1832930245,v1:127.0.0.1:6803/1832930245] [v2:127.0.0.1:6804/1832930245,v1:127.0.0.1:6805/1832930245] exists,up b706816b-c6ef-45c0-a473-57c73761fb52 2026-02-17T11:26:02.663 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 41364ef8-bd7c-4975-bb7f-b2dcfdb0f46e 2026-02-17T11:26:02.829 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:26:02.854 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:02.847+0000 7fbff9d0e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:26:02.855 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:02.847+0000 7fbff9d0e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:26:02.856 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:02.851+0000 7fbff9d0e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:26:02.857 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:02.851+0000 7fbff9d0e8c0 -1 bdev(0x55b29adc3c00 td/osd-mapper/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:26:02.857 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:02.851+0000 7fbff9d0e8c0 -1 bluestore(td/osd-mapper/1) _read_fsid unparsable uuid 2026-02-17T11:26:05.117 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:26:05.306 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:26:05.331 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:05.323+0000 7fa4812e68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:26:05.333 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:05.327+0000 7fa4812e68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:26:05.334 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:05.327+0000 7fa4812e68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:26:05.461 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:26:06.614 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:26:06.833 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:06.827+0000 7fa4812e68c0 -1 Falling back to public interface 2026-02-17T11:26:07.760 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:26:08.909 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:26:09.826 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:09.819+0000 7fa4812e68c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:26:10.056 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:26:11.220 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:26:11.369 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4253974934,v1:127.0.0.1:6811/4253974934] [v2:127.0.0.1:6812/4253974934,v1:127.0.0.1:6813/4253974934] exists,up 41364ef8-bd7c-4975-bb7f-b2dcfdb0f46e 2026-02-17T11:26:11.374 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 ee2c5b75-fce6-43eb-8fa3-50506aad4231 2026-02-17T11:26:11.547 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:26:11.574 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:11.567+0000 7f328fd2b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:26:11.575 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:11.567+0000 7f328fd2b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:26:11.576 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:11.571+0000 7f328fd2b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:26:11.577 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:11.571+0000 7f328fd2b8c0 -1 bdev(0x559d2c019c00 td/osd-mapper/2/block) open stat got: (1) Operation not permitted 2026-02-17T11:26:11.577 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:11.571+0000 7f328fd2b8c0 -1 bluestore(td/osd-mapper/2) _read_fsid unparsable uuid 2026-02-17T11:26:13.841 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T11:26:14.040 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T11:26:14.057 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:14.051+0000 7f7509a7e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:26:14.058 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:14.051+0000 7f7509a7e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:26:14.059 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:14.051+0000 7f7509a7e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:26:14.192 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:26:15.346 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:26:15.349 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:15.343+0000 7f7509a7e8c0 -1 Falling back to public interface 2026-02-17T11:26:16.501 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:26:17.657 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:26:18.558 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:18.551+0000 7f7509a7e8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T11:26:18.812 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:26:19.966 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:26:20.119 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2186276671,v1:127.0.0.1:6819/2186276671] [v2:127.0.0.1:6820/2186276671,v1:127.0.0.1:6821/2186276671] exists,up ee2c5b75-fce6-43eb-8fa3-50506aad4231 2026-02-17T11:26:20.319 INFO:tasks.workunit.client.0.trial128.stderr:pool 'test' created 2026-02-17T11:26:21.794 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836485 2026-02-17T11:26:23.108 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705668 2026-02-17T11:26:23.275 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574850 2026-02-17T11:26:25.077 INFO:tasks.workunit.client.0.trial128.stdout:standard_scrub_cluster: dbg: test pool is test 1 2026-02-17T11:26:25.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:55: TEST_truncated_sna_record: ceph tell 'osd.*' config set osd_stats_update_period_not_scrubbing 1 2026-02-17T11:26:25.142 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T11:26:25.142 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_stats_update_period_not_scrubbing = '' (not observed, change may require restart) " 2026-02-17T11:26:25.143 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:26:25.149 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T11:26:25.149 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_stats_update_period_not_scrubbing = '' (not observed, change may require restart) " 2026-02-17T11:26:25.149 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:26:25.155 INFO:tasks.workunit.client.0.trial128.stdout:osd.2: { 2026-02-17T11:26:25.155 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_stats_update_period_not_scrubbing = '' (not observed, change may require restart) " 2026-02-17T11:26:25.155 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:26:25.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:56: TEST_truncated_sna_record: ceph tell 'osd.*' config set osd_stats_update_period_scrubbing 1 2026-02-17T11:26:25.238 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T11:26:25.239 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_stats_update_period_scrubbing = '' (not observed, change may require restart) " 2026-02-17T11:26:25.239 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:26:25.245 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T11:26:25.245 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_stats_update_period_scrubbing = '' (not observed, change may require restart) " 2026-02-17T11:26:25.245 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:26:25.250 INFO:tasks.workunit.client.0.trial128.stdout:osd.2: { 2026-02-17T11:26:25.251 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_stats_update_period_scrubbing = '' (not observed, change may require restart) " 2026-02-17T11:26:25.251 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:26:25.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:58: TEST_truncated_sna_record: local osdn=3 2026-02-17T11:26:25.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:59: TEST_truncated_sna_record: local poolid=1 2026-02-17T11:26:25.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:60: TEST_truncated_sna_record: local poolname=test 2026-02-17T11:26:25.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:61: TEST_truncated_sna_record: local objname=objxxx 2026-02-17T11:26:25.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:64: TEST_truncated_sna_record: make_a_clone test objxxx snap01 snap02 2026-02-17T11:26:25.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:32: make_a_clone: local saved_echo_flag=x 2026-02-17T11:26:25.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:33: make_a_clone: set +x 2026-02-17T11:26:25.322 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap01 2026-02-17T11:26:25.424 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap02 2026-02-17T11:26:25.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:65: TEST_truncated_sna_record: make_a_clone test objxxx snap13 2026-02-17T11:26:25.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:32: make_a_clone: local saved_echo_flag=x 2026-02-17T11:26:25.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:33: make_a_clone: set +x 2026-02-17T11:26:25.526 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap13 2026-02-17T11:26:25.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:66: TEST_truncated_sna_record: make_a_clone test objxxx snap24 snap25 2026-02-17T11:26:25.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:32: make_a_clone: local saved_echo_flag=x 2026-02-17T11:26:25.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:33: make_a_clone: set +x 2026-02-17T11:26:25.629 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap24 2026-02-17T11:26:25.730 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap25 2026-02-17T11:26:25.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:67: TEST_truncated_sna_record: echo 6959 2026-02-17T11:26:25.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:67: TEST_truncated_sna_record: rados -p test put objxxx - 2026-02-17T11:26:25.755 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:70: TEST_truncated_sna_record: ceph --format=json-pretty osd map test objxxx 2026-02-17T11:26:25.755 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:70: TEST_truncated_sna_record: jq -r .pgid 2026-02-17T11:26:25.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:70: TEST_truncated_sna_record: local pgid=1.3 2026-02-17T11:26:25.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:71: TEST_truncated_sna_record: ceph --format=json-pretty osd map test objxxx 2026-02-17T11:26:25.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:71: TEST_truncated_sna_record: jq -r '.up[0]' 2026-02-17T11:26:26.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:71: TEST_truncated_sna_record: local osd=1 2026-02-17T11:26:26.052 INFO:tasks.workunit.client.0.trial128.stdout:pgid is 1.3 (primary: osd.1) 2026-02-17T11:26:26.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:72: TEST_truncated_sna_record: echo 'pgid is 1.3 (primary: osd.1)' 2026-02-17T11:26:26.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:74: TEST_truncated_sna_record: set_query_debug 1.3 2026-02-17T11:26:26.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:303: set_query_debug: local pgid=1.3 2026-02-17T11:26:26.052 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:305: set_query_debug: ceph pg dump pgs_brief 2026-02-17T11:26:26.052 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:305: set_query_debug: awk -v 'pg=^1.3' -n -e '$0 ~ pg { print(gensub(/[^0-9]*([0-9]+).*/,"\\1","g",$5)); }' 2026-02-17T11:26:26.192 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs_brief 2026-02-17T11:26:26.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:305: set_query_debug: local prim_osd=1 2026-02-17T11:26:26.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:307: set_query_debug: echo 'Setting scrub debug data. Primary for 1.3 is 1' 2026-02-17T11:26:26.201 INFO:tasks.workunit.client.0.trial128.stdout:Setting scrub debug data. Primary for 1.3 is 1 2026-02-17T11:26:26.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:308: set_query_debug: get_asok_path osd.1 2026-02-17T11:26:26.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T11:26:26.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T11:26:26.202 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:26:26.202 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:26:26.202 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11816 2026-02-17T11:26:26.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.11816/ceph-osd.1.asok 2026-02-17T11:26:26.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:308: set_query_debug: CEPH_ARGS= 2026-02-17T11:26:26.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:308: set_query_debug: ceph --format=json daemon /tmp/ceph-asok.11816/ceph-osd.1.asok scrubdebug 1.3 set sessions 2026-02-17T11:26:26.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:77: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T11:26:26.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:77: TEST_truncated_sna_record: rados --format json-pretty -p test listsnaps objxxx 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout:{"success":true}{ 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: "name": "objxxx", 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 5, 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: "id": 2, 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: "snapshots": [ 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: "id": 1, 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: "name": "snap01" 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: "id": 2, 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: "name": "snap02" 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: "size": 6, 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: "overlaps": [] 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: "id": 3, 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: "snapshots": [ 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: "id": 3, 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: "name": "snap13" 2026-02-17T11:26:26.295 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: "size": 6, 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: "overlaps": [] 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: "id": 5, 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: "snapshots": [ 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: "id": 4, 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: "name": "snap24" 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: "id": 5, 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: "name": "snap25" 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: "size": 6, 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: "overlaps": [] 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: "id": "head", 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: "snapshots": [], 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: "size": 5 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:26:26.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:80: TEST_truncated_sna_record: ceph pg 1.3 deep-scrub 2026-02-17T11:26:26.363 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T11:26:26.363 INFO:tasks.workunit.client.0.trial128.stdout: "deep": true, 2026-02-17T11:26:26.363 INFO:tasks.workunit.client.0.trial128.stdout: "must": true, 2026-02-17T11:26:26.363 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "0.000000" 2026-02-17T11:26:26.363 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:26:26.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:84: TEST_truncated_sna_record: sleep 3 2026-02-17T11:26:29.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:85: TEST_truncated_sna_record: ceph pg dump pgs 2026-02-17T11:26:29.515 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T11:26:29.516 INFO:tasks.workunit.client.0.trial128.stdout:1.3 4 0 0 0 0 23 0 0 7 0 7 active+clean 2026-02-17T11:26:20.425347+0000 21'7 21:31 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:26:20.315961+0000 0 0 2026-02-17T11:26:29.516 INFO:tasks.workunit.client.0.trial128.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T11:26:20.424784+0000 0'0 21:25 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:26:20.315961+0000 0 0 2026-02-17T11:26:29.516 INFO:tasks.workunit.client.0.trial128.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T11:26:20.424382+0000 0'0 16:13 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:26:20.315961+0000 0 0 2026-02-17T11:26:29.516 INFO:tasks.workunit.client.0.trial128.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T11:26:20.425339+0000 0'0 21:27 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:26:20.315961+0000 0 0 2026-02-17T11:26:29.516 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:29.516 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T11:26:29.516 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:26:29.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:86: TEST_truncated_sna_record: grep -a -q -- 'event: --^^^^---- ScrubFinished' td/osd-mapper/osd.1.log 2026-02-17T11:26:29.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:90: TEST_truncated_sna_record: ceph pg dump pgs 2026-02-17T11:26:29.669 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T11:26:29.669 INFO:tasks.workunit.client.0.trial128.stdout:1.3 4 0 0 0 0 23 0 0 7 0 7 active+clean 2026-02-17T11:26:20.425347+0000 21'7 21:31 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:26:20.315961+0000 0 0 2026-02-17T11:26:29.670 INFO:tasks.workunit.client.0.trial128.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T11:26:20.424784+0000 0'0 21:25 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:26:20.315961+0000 0 0 2026-02-17T11:26:29.670 INFO:tasks.workunit.client.0.trial128.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T11:26:20.424382+0000 0'0 16:13 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:26:20.315961+0000 0 0 2026-02-17T11:26:29.670 INFO:tasks.workunit.client.0.trial128.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T11:26:20.425339+0000 0'0 21:27 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:26:20.315961+0000 0 0 2026-02-17T11:26:29.670 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:29.670 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T11:26:29.670 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:26:29.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:91: TEST_truncated_sna_record: ceph osd set noscrub 2026-02-17T11:26:29.865 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:26:29.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:92: TEST_truncated_sna_record: ceph osd set nodeep-scrub 2026-02-17T11:26:30.065 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:26:30.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:93: TEST_truncated_sna_record: sleep 5 2026-02-17T11:26:35.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:94: TEST_truncated_sna_record: grep -a -q -v ERR td/osd-mapper/osd.1.log 2026-02-17T11:26:35.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:97: TEST_truncated_sna_record: kill_daemons td/osd-mapper TERM osd 2026-02-17T11:26:35.079 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:26:35.079 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:26:35.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:26:35.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:26:35.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:26:36.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:26:36.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:99: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T11:26:36.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:99: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 dump p 2026-02-17T11:26:36.405 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.398+0000 7f696c6b2bc0 1 bdev(0x55941861b800 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-17T11:26:36.405 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.398+0000 7f696c6b2bc0 1 bdev(0x55941861b800 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:36.406 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.398+0000 7f696c6b2bc0 1 bdev(0x55941861b800 td/osd-mapper/0/block) close 2026-02-17T11:26:36.684 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.678+0000 7f696c6b2bc0 1 bdev(0x55941861b800 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-17T11:26:36.684 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.678+0000 7f696c6b2bc0 1 bdev(0x55941861b800 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:36.685 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.678+0000 7f696c6b2bc0 1 bluestore(td/osd-mapper/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T11:26:36.685 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.678+0000 7f696c6b2bc0 1 bdev(0x55941861bc00 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-17T11:26:36.685 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.678+0000 7f696c6b2bc0 1 bdev(0x55941861bc00 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:36.685 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.678+0000 7f696c6b2bc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/0/block size 100 GiB 2026-02-17T11:26:36.685 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.678+0000 7f696c6b2bc0 1 bluefs mount 2026-02-17T11:26:36.685 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.678+0000 7f696c6b2bc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T11:26:36.688 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.678+0000 7f696c6b2bc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:36.688 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.678+0000 7f696c6b2bc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:36.688 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.678+0000 7f696c6b2bc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:36.688 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.678+0000 7f696c6b2bc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:36.688 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.678+0000 7f696c6b2bc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:36.688 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.678+0000 7f696c6b2bc0 1 bluefs mount shared_bdev_used = 0 2026-02-17T11:26:36.688 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.678+0000 7f696c6b2bc0 1 bluestore(td/osd-mapper/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T11:26:36.701 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.694+0000 7f696c6b2bc0 1 bluestore(td/osd-mapper/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T11:26:36.701 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.694+0000 7f696c6b2bc0 1 bluestore(td/osd-mapper/0) _open_super_meta old nid_max 2051 2026-02-17T11:26:36.701 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.694+0000 7f696c6b2bc0 1 bluestore(td/osd-mapper/0) _open_super_meta old blobid_max 20480 2026-02-17T11:26:36.701 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.694+0000 7f696c6b2bc0 1 bluestore(td/osd-mapper/0) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T11:26:36.701 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.694+0000 7f696c6b2bc0 1 bluestore(td/osd-mapper/0) _open_super_meta min_alloc_size 0x1000 2026-02-17T11:26:36.702 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.694+0000 7f696c6b2bc0 1 freelist init 2026-02-17T11:26:36.702 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.694+0000 7f696c6b2bc0 1 freelist _read_cfg 2026-02-17T11:26:36.703 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.694+0000 7f696c6b2bc0 1 bluestore(td/osd-mapper/0) _init_alloc loaded 100 GiB in 4 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffca000, fragmentation 1.14441e-07 2026-02-17T11:26:36.703 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.694+0000 7f696c6b2bc0 1 bluefs umount 2026-02-17T11:26:36.703 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.694+0000 7f696c6b2bc0 1 bdev(0x55941861bc00 td/osd-mapper/0/block) close 2026-02-17T11:26:36.964 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.958+0000 7f696c6b2bc0 1 bdev(0x55941861bc00 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-17T11:26:36.964 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.958+0000 7f696c6b2bc0 1 bdev(0x55941861bc00 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:36.965 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.958+0000 7f696c6b2bc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/0/block size 100 GiB 2026-02-17T11:26:36.965 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.958+0000 7f696c6b2bc0 1 bluefs mount 2026-02-17T11:26:36.965 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.958+0000 7f696c6b2bc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T11:26:36.965 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.958+0000 7f696c6b2bc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:36.965 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.958+0000 7f696c6b2bc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:36.965 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.958+0000 7f696c6b2bc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:36.965 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.958+0000 7f696c6b2bc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:36.965 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.958+0000 7f696c6b2bc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:36.965 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.958+0000 7f696c6b2bc0 1 bluefs mount shared_bdev_used = 7864320 2026-02-17T11:26:36.966 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.958+0000 7f696c6b2bc0 1 bluestore(td/osd-mapper/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T11:26:36.978 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.970+0000 7f696c6b2bc0 1 bluestore(td/osd-mapper/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000000 0b 05 58 02 00 00 fd b2 54 7e 2a 2a 41 a0 b1 69 |..X.....T~**A..i| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000010 5d fc 68 2b 2f fb 00 00 00 00 17 00 00 00 00 00 |].h+/...........| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:* 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2026-02-17T11:26:36.979 INFO:tasks.workunit.client.0.trial128.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000220 72 65 17 00 00 00 05 00 00 00 b7 06 81 6b c6 ef |re...........k..| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000230 45 c0 a4 73 57 c7 37 61 fb 52 00 00 00 00 00 00 |E..sW.7a.R......| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000240 00 00 17 00 00 00 4a 50 94 69 be 0b 41 03 01 00 |......JP.i..A...| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000250 00 00 01 00 00 00 01 00 00 00 17 00 00 00 |..............| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:0000025e 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000047 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:0000003f 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T11:26:36.980 INFO:tasks.workunit.client.0.trial128.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:00000047 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:36.981 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:36.982 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:36.982 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:36.982 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:36.982 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:36.982 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:36.982 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:36.983 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:36.983 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:36.983 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:36.983 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:36.983 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:36.983 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:36.983 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:36.983 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:36.983 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:36.983 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:36.984 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:36.984 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:36.984 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:36.984 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2026-02-17T11:26:36.984 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:36.985 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2026-02-17T11:26:36.985 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:36.985 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.970+0000 7f696c6b2bc0 1 bluefs umount 2026-02-17T11:26:36.985 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:36.970+0000 7f696c6b2bc0 1 bdev(0x55941861bc00 td/osd-mapper/0/block) close 2026-02-17T11:26:37.244 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.238+0000 7f696c6b2bc0 1 freelist shutdown 2026-02-17T11:26:37.245 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.238+0000 7f696c6b2bc0 1 bdev(0x55941861b800 td/osd-mapper/0/block) close 2026-02-17T11:26:37.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:100: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T11:26:37.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:100: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 dump p 2026-02-17T11:26:37.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:100: TEST_truncated_sna_record: grep -a SNA_ 2026-02-17T11:26:37.491 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.482+0000 7efff091ebc0 1 bdev(0x55f2383ab800 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-17T11:26:37.491 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.482+0000 7efff091ebc0 1 bdev(0x55f2383ab800 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:37.492 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.482+0000 7efff091ebc0 1 bdev(0x55f2383ab800 td/osd-mapper/2/block) close 2026-02-17T11:26:37.773 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.766+0000 7efff091ebc0 1 bdev(0x55f2383ab800 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-17T11:26:37.773 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.766+0000 7efff091ebc0 1 bdev(0x55f2383ab800 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:37.774 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.766+0000 7efff091ebc0 1 bluestore(td/osd-mapper/2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T11:26:37.774 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.766+0000 7efff091ebc0 1 bdev(0x55f2383abc00 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-17T11:26:37.774 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.766+0000 7efff091ebc0 1 bdev(0x55f2383abc00 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:37.774 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.766+0000 7efff091ebc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/2/block size 100 GiB 2026-02-17T11:26:37.774 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.766+0000 7efff091ebc0 1 bluefs mount 2026-02-17T11:26:37.774 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.766+0000 7efff091ebc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T11:26:37.776 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.766+0000 7efff091ebc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:37.776 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.766+0000 7efff091ebc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:37.776 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.766+0000 7efff091ebc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:37.776 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.766+0000 7efff091ebc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:37.776 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.766+0000 7efff091ebc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:37.776 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.766+0000 7efff091ebc0 1 bluefs mount shared_bdev_used = 0 2026-02-17T11:26:37.776 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.766+0000 7efff091ebc0 1 bluestore(td/osd-mapper/2) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T11:26:37.798 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.790+0000 7efff091ebc0 1 bluestore(td/osd-mapper/2) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T11:26:37.798 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.790+0000 7efff091ebc0 1 bluestore(td/osd-mapper/2) _open_super_meta old nid_max 2050 2026-02-17T11:26:37.798 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.790+0000 7efff091ebc0 1 bluestore(td/osd-mapper/2) _open_super_meta old blobid_max 20480 2026-02-17T11:26:37.798 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.790+0000 7efff091ebc0 1 bluestore(td/osd-mapper/2) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T11:26:37.799 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.790+0000 7efff091ebc0 1 bluestore(td/osd-mapper/2) _open_super_meta min_alloc_size 0x1000 2026-02-17T11:26:37.799 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.790+0000 7efff091ebc0 1 freelist init 2026-02-17T11:26:37.799 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.790+0000 7efff091ebc0 1 freelist _read_cfg 2026-02-17T11:26:37.801 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.790+0000 7efff091ebc0 1 bluestore(td/osd-mapper/2) _init_alloc loaded 100 GiB in 5 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffca000, fragmentation 1.52588e-07 2026-02-17T11:26:37.801 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.790+0000 7efff091ebc0 1 bluefs umount 2026-02-17T11:26:37.801 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:37.790+0000 7efff091ebc0 1 bdev(0x55f2383abc00 td/osd-mapper/2/block) close 2026-02-17T11:26:38.057 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.050+0000 7efff091ebc0 1 bdev(0x55f2383abc00 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-17T11:26:38.057 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.050+0000 7efff091ebc0 1 bdev(0x55f2383abc00 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:38.057 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.050+0000 7efff091ebc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/2/block size 100 GiB 2026-02-17T11:26:38.057 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.050+0000 7efff091ebc0 1 bluefs mount 2026-02-17T11:26:38.057 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.050+0000 7efff091ebc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T11:26:38.058 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.050+0000 7efff091ebc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:38.058 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.050+0000 7efff091ebc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:38.058 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.050+0000 7efff091ebc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:38.058 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.050+0000 7efff091ebc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:38.058 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.050+0000 7efff091ebc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:38.058 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.050+0000 7efff091ebc0 1 bluefs mount shared_bdev_used = 7864320 2026-02-17T11:26:38.059 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.050+0000 7efff091ebc0 1 bluestore(td/osd-mapper/2) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T11:26:38.072 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.062+0000 7efff091ebc0 1 bluestore(td/osd-mapper/2) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T11:26:38.072 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.062+0000 7efff091ebc0 1 bluefs umount 2026-02-17T11:26:38.072 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.062+0000 7efff091ebc0 1 bdev(0x55f2383abc00 td/osd-mapper/2/block) close 2026-02-17T11:26:38.336 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.330+0000 7efff091ebc0 1 freelist shutdown 2026-02-17T11:26:38.336 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:38.330+0000 7efff091ebc0 1 bdev(0x55f2383ab800 td/osd-mapper/2/block) close 2026-02-17T11:26:38.564 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:38.565 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:38.565 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:38.565 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:38.565 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:38.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:101: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T11:26:38.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:101: TEST_truncated_sna_record: grep -a SNA_ /tmp/oo2.dump 2026-02-17T11:26:38.566 INFO:tasks.workunit.client.0.trial128.stderr:grep: /tmp/oo2.dump: No such file or directory 2026-02-17T11:26:38.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:102: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T11:26:38.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:102: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 dump p 2026-02-17T11:26:39.158 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2026-02-17T11:26:39.158 INFO:tasks.workunit.client.0.trial128.stdout:00000000 0b 05 58 02 00 00 fd b2 54 7e 2a 2a 41 a0 b1 69 |..X.....T~**A..i| 2026-02-17T11:26:39.158 INFO:tasks.workunit.client.0.trial128.stdout:00000010 5d fc 68 2b 2f fb 02 00 00 00 17 00 00 00 00 00 |].h+/...........| 2026-02-17T11:26:39.158 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2026-02-17T11:26:39.158 INFO:tasks.workunit.client.0.trial128.stdout:* 2026-02-17T11:26:39.158 INFO:tasks.workunit.client.0.trial128.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2026-02-17T11:26:39.158 INFO:tasks.workunit.client.0.trial128.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2026-02-17T11:26:39.158 INFO:tasks.workunit.client.0.trial128.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000220 72 65 17 00 00 00 0d 00 00 00 ee 2c 5b 75 fc e6 |re.........,[u..| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000230 43 eb 8f a3 50 50 6a ad 42 31 00 00 00 00 00 00 |C...PPj.B1......| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000240 00 00 17 00 00 00 5b 50 94 69 6a ca 18 21 01 00 |......[P.ij..!..| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000250 00 00 01 00 00 00 01 00 00 00 17 00 00 00 |..............| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:0000025e 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000047 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:39.159 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:0000003f 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000047 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:39.160 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:39.161 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:39.161 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:39.161 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:39.161 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:39.161 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2026-02-17T11:26:39.161 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:39.161 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2026-02-17T11:26:39.161 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:39.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2026-02-17T11:26:39.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_000000000000000[0-9]_000000000000000' 2026-02-17T11:26:39.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-17T11:26:39.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: wc -l 2026-02-17T11:26:40.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:104: TEST_truncated_sna_record: local num_sna_b4=5 2026-02-17T11:26:40.740 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: expr 3 - 1 2026-02-17T11:26:40.740 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: seq 0 2 2026-02-17T11:26:40.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T11:26:40.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:108: TEST_truncated_sna_record: kvdir=td/osd-mapper/0 2026-02-17T11:26:40.743 INFO:tasks.workunit.client.0.trial128.stdout:corrupting the SnapMapper DB of osd.0 (db: td/osd-mapper/0) 2026-02-17T11:26:40.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:109: TEST_truncated_sna_record: echo 'corrupting the SnapMapper DB of osd.0 (db: td/osd-mapper/0)' 2026-02-17T11:26:40.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: (( extr_dbg >= 3 )) 2026-02-17T11:26:40.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 dump p 2026-02-17T11:26:40.756 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:40.746+0000 7efe41f53bc0 1 bdev(0x55bb17b1b800 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-17T11:26:40.756 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:40.746+0000 7efe41f53bc0 1 bdev(0x55bb17b1b800 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:40.756 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:40.746+0000 7efe41f53bc0 1 bdev(0x55bb17b1b800 td/osd-mapper/0/block) close 2026-02-17T11:26:41.036 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.030+0000 7efe41f53bc0 1 bdev(0x55bb17b1b800 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-17T11:26:41.036 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.030+0000 7efe41f53bc0 1 bdev(0x55bb17b1b800 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:41.037 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.030+0000 7efe41f53bc0 1 bluestore(td/osd-mapper/0) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T11:26:41.037 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.030+0000 7efe41f53bc0 1 bdev(0x55bb17b1bc00 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-17T11:26:41.037 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.030+0000 7efe41f53bc0 1 bdev(0x55bb17b1bc00 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:41.037 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.030+0000 7efe41f53bc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/0/block size 100 GiB 2026-02-17T11:26:41.037 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.030+0000 7efe41f53bc0 1 bluefs mount 2026-02-17T11:26:41.037 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.030+0000 7efe41f53bc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T11:26:41.040 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.030+0000 7efe41f53bc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:41.040 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.030+0000 7efe41f53bc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:41.040 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.030+0000 7efe41f53bc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:41.040 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.030+0000 7efe41f53bc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:41.040 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.030+0000 7efe41f53bc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:41.040 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.030+0000 7efe41f53bc0 1 bluefs mount shared_bdev_used = 0 2026-02-17T11:26:41.040 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.034+0000 7efe41f53bc0 1 bluestore(td/osd-mapper/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T11:26:41.062 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.054+0000 7efe41f53bc0 1 bluestore(td/osd-mapper/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T11:26:41.062 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.054+0000 7efe41f53bc0 1 bluestore(td/osd-mapper/0) _open_super_meta old nid_max 2051 2026-02-17T11:26:41.062 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.054+0000 7efe41f53bc0 1 bluestore(td/osd-mapper/0) _open_super_meta old blobid_max 20480 2026-02-17T11:26:41.062 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.054+0000 7efe41f53bc0 1 bluestore(td/osd-mapper/0) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T11:26:41.062 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.054+0000 7efe41f53bc0 1 bluestore(td/osd-mapper/0) _open_super_meta min_alloc_size 0x1000 2026-02-17T11:26:41.063 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.054+0000 7efe41f53bc0 1 freelist init 2026-02-17T11:26:41.063 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.054+0000 7efe41f53bc0 1 freelist _read_cfg 2026-02-17T11:26:41.065 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.058+0000 7efe41f53bc0 1 bluestore(td/osd-mapper/0) _init_alloc loaded 100 GiB in 4 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffca000, fragmentation 1.14441e-07 2026-02-17T11:26:41.065 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.058+0000 7efe41f53bc0 1 bluefs umount 2026-02-17T11:26:41.065 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.058+0000 7efe41f53bc0 1 bdev(0x55bb17b1bc00 td/osd-mapper/0/block) close 2026-02-17T11:26:41.320 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.314+0000 7efe41f53bc0 1 bdev(0x55bb17b1bc00 td/osd-mapper/0/block) open path td/osd-mapper/0/block 2026-02-17T11:26:41.322 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.314+0000 7efe41f53bc0 1 bdev(0x55bb17b1bc00 td/osd-mapper/0/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:41.322 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.314+0000 7efe41f53bc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/0/block size 100 GiB 2026-02-17T11:26:41.322 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.314+0000 7efe41f53bc0 1 bluefs mount 2026-02-17T11:26:41.323 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.314+0000 7efe41f53bc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T11:26:41.323 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.314+0000 7efe41f53bc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:41.323 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.314+0000 7efe41f53bc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:41.323 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.314+0000 7efe41f53bc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:41.323 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.314+0000 7efe41f53bc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:41.324 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.314+0000 7efe41f53bc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:41.324 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.314+0000 7efe41f53bc0 1 bluefs mount shared_bdev_used = 7864320 2026-02-17T11:26:41.324 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.314+0000 7efe41f53bc0 1 bluestore(td/osd-mapper/0) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T11:26:41.335 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.326+0000 7efe41f53bc0 1 bluestore(td/osd-mapper/0) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T11:26:41.336 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2026-02-17T11:26:41.336 INFO:tasks.workunit.client.0.trial128.stdout:00000000 0b 05 58 02 00 00 fd b2 54 7e 2a 2a 41 a0 b1 69 |..X.....T~**A..i| 2026-02-17T11:26:41.336 INFO:tasks.workunit.client.0.trial128.stdout:00000010 5d fc 68 2b 2f fb 00 00 00 00 17 00 00 00 00 00 |].h+/...........| 2026-02-17T11:26:41.336 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2026-02-17T11:26:41.337 INFO:tasks.workunit.client.0.trial128.stdout:* 2026-02-17T11:26:41.337 INFO:tasks.workunit.client.0.trial128.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2026-02-17T11:26:41.337 INFO:tasks.workunit.client.0.trial128.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2026-02-17T11:26:41.337 INFO:tasks.workunit.client.0.trial128.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2026-02-17T11:26:41.337 INFO:tasks.workunit.client.0.trial128.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2026-02-17T11:26:41.337 INFO:tasks.workunit.client.0.trial128.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2026-02-17T11:26:41.337 INFO:tasks.workunit.client.0.trial128.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2026-02-17T11:26:41.337 INFO:tasks.workunit.client.0.trial128.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2026-02-17T11:26:41.337 INFO:tasks.workunit.client.0.trial128.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2026-02-17T11:26:41.337 INFO:tasks.workunit.client.0.trial128.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2026-02-17T11:26:41.337 INFO:tasks.workunit.client.0.trial128.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2026-02-17T11:26:41.337 INFO:tasks.workunit.client.0.trial128.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2026-02-17T11:26:41.338 INFO:tasks.workunit.client.0.trial128.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2026-02-17T11:26:41.338 INFO:tasks.workunit.client.0.trial128.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2026-02-17T11:26:41.338 INFO:tasks.workunit.client.0.trial128.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2026-02-17T11:26:41.338 INFO:tasks.workunit.client.0.trial128.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2026-02-17T11:26:41.338 INFO:tasks.workunit.client.0.trial128.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2026-02-17T11:26:41.338 INFO:tasks.workunit.client.0.trial128.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2026-02-17T11:26:41.338 INFO:tasks.workunit.client.0.trial128.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2026-02-17T11:26:41.338 INFO:tasks.workunit.client.0.trial128.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2026-02-17T11:26:41.338 INFO:tasks.workunit.client.0.trial128.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2026-02-17T11:26:41.338 INFO:tasks.workunit.client.0.trial128.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2026-02-17T11:26:41.338 INFO:tasks.workunit.client.0.trial128.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2026-02-17T11:26:41.338 INFO:tasks.workunit.client.0.trial128.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2026-02-17T11:26:41.339 INFO:tasks.workunit.client.0.trial128.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2026-02-17T11:26:41.339 INFO:tasks.workunit.client.0.trial128.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2026-02-17T11:26:41.339 INFO:tasks.workunit.client.0.trial128.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2026-02-17T11:26:41.339 INFO:tasks.workunit.client.0.trial128.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2026-02-17T11:26:41.339 INFO:tasks.workunit.client.0.trial128.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2026-02-17T11:26:41.339 INFO:tasks.workunit.client.0.trial128.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2026-02-17T11:26:41.339 INFO:tasks.workunit.client.0.trial128.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2026-02-17T11:26:41.339 INFO:tasks.workunit.client.0.trial128.stdout:00000220 72 65 17 00 00 00 05 00 00 00 b7 06 81 6b c6 ef |re...........k..| 2026-02-17T11:26:41.339 INFO:tasks.workunit.client.0.trial128.stdout:00000230 45 c0 a4 73 57 c7 37 61 fb 52 00 00 00 00 00 00 |E..sW.7a.R......| 2026-02-17T11:26:41.339 INFO:tasks.workunit.client.0.trial128.stdout:00000240 00 00 17 00 00 00 4a 50 94 69 be 0b 41 03 01 00 |......JP.i..A...| 2026-02-17T11:26:41.339 INFO:tasks.workunit.client.0.trial128.stdout:00000250 00 00 01 00 00 00 01 00 00 00 17 00 00 00 |..............| 2026-02-17T11:26:41.339 INFO:tasks.workunit.client.0.trial128.stdout:0000025e 2026-02-17T11:26:41.339 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:41.340 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2026-02-17T11:26:41.340 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:41.340 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:41.340 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T11:26:41.340 INFO:tasks.workunit.client.0.trial128.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2026-02-17T11:26:41.340 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T11:26:41.340 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2026-02-17T11:26:41.340 INFO:tasks.workunit.client.0.trial128.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T11:26:41.340 INFO:tasks.workunit.client.0.trial128.stdout:00000047 2026-02-17T11:26:41.340 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:41.340 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:41.341 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2026-02-17T11:26:41.341 INFO:tasks.workunit.client.0.trial128.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2026-02-17T11:26:41.341 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T11:26:41.341 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2026-02-17T11:26:41.341 INFO:tasks.workunit.client.0.trial128.stdout:0000003f 2026-02-17T11:26:41.341 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:41.341 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:41.341 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T11:26:41.341 INFO:tasks.workunit.client.0.trial128.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2026-02-17T11:26:41.341 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T11:26:41.341 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2026-02-17T11:26:41.341 INFO:tasks.workunit.client.0.trial128.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T11:26:41.342 INFO:tasks.workunit.client.0.trial128.stdout:00000047 2026-02-17T11:26:41.342 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:41.342 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:41.342 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:41.342 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:41.342 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:41.342 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:41.342 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:41.342 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:41.342 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:41.342 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:41.342 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:41.343 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:41.343 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:41.343 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:41.343 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:41.343 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:41.343 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:41.343 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:41.343 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:41.343 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:41.343 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:41.343 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:41.343 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:41.343 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:41.344 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:41.344 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:41.344 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:41.344 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:41.344 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:41.344 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:41.344 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:41.344 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:41.344 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:41.344 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:41.344 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:41.344 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:41.345 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2026-02-17T11:26:41.345 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:41.345 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2026-02-17T11:26:41.345 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:41.345 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.326+0000 7efe41f53bc0 1 bluefs umount 2026-02-17T11:26:41.345 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.326+0000 7efe41f53bc0 1 bdev(0x55bb17b1bc00 td/osd-mapper/0/block) close 2026-02-17T11:26:41.600 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.594+0000 7efe41f53bc0 1 freelist shutdown 2026-02-17T11:26:41.601 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:41.594+0000 7efe41f53bc0 1 bdev(0x55bb17b1b800 td/osd-mapper/0/block) close 2026-02-17T11:26:41.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 dump p 2026-02-17T11:26:41.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_0000000000000003_000000000000000' 2026-02-17T11:26:41.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-17T11:26:42.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: KY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:42.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T11:26:42.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: echo 'SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..' 2026-02-17T11:26:42.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: cat -v 2026-02-17T11:26:42.903 INFO:tasks.workunit.client.0.trial128.stdout:SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:42.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: mktemp -p /tmp --suffix=_the_val 2026-02-17T11:26:42.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: tmp_fn1=/tmp/tmp.idRp0dFf4v_the_val 2026-02-17T11:26:42.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T11:26:42.905 INFO:tasks.workunit.client.0.trial128.stdout:Value dumped in: /tmp/tmp.idRp0dFf4v_the_val 2026-02-17T11:26:42.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: echo 'Value dumped in: /tmp/tmp.idRp0dFf4v_the_val' 2026-02-17T11:26:42.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:119: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 get p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. out /tmp/tmp.idRp0dFf4v_the_val 2026-02-17T11:26:43.503 INFO:tasks.workunit.client.0.trial128.stdout:(p, %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..) 2026-02-17T11:26:43.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T11:26:43.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: od -xc /tmp/tmp.idRp0dFf4v_the_val 2026-02-17T11:26:43.986 INFO:tasks.workunit.client.0.trial128.stdout:0000000 0101 0035 0000 0003 0000 0000 0000 0304 2026-02-17T11:26:43.986 INFO:tasks.workunit.client.0.trial128.stdout: 001 001 5 \0 \0 \0 003 \0 \0 \0 \0 \0 \0 \0 004 003 2026-02-17T11:26:43.986 INFO:tasks.workunit.client.0.trial128.stdout:0000020 0027 0000 0000 0000 0006 0000 626f 786a 2026-02-17T11:26:43.986 INFO:tasks.workunit.client.0.trial128.stdout: ' \0 \0 \0 \0 \0 \0 \0 006 \0 \0 \0 o b j x 2026-02-17T11:26:43.986 INFO:tasks.workunit.client.0.trial128.stdout:0000040 7878 0003 0000 0000 0000 75af 85d0 0000 2026-02-17T11:26:43.986 INFO:tasks.workunit.client.0.trial128.stdout: x x 003 \0 \0 \0 \0 \0 \0 \0 257 u 320 205 \0 \0 2026-02-17T11:26:43.986 INFO:tasks.workunit.client.0.trial128.stdout:0000060 0000 0100 0000 0000 0000 0000 2026-02-17T11:26:43.986 INFO:tasks.workunit.client.0.trial128.stdout: \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0 2026-02-17T11:26:43.986 INFO:tasks.workunit.client.0.trial128.stdout:0000073 2026-02-17T11:26:43.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:122: TEST_truncated_sna_record: NKY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 2026-02-17T11:26:43.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:123: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 rm p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:45.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:124: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/0 set p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 in /tmp/tmp.idRp0dFf4v_the_val 2026-02-17T11:26:46.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:126: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T11:26:46.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T11:26:46.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:108: TEST_truncated_sna_record: kvdir=td/osd-mapper/1 2026-02-17T11:26:46.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:109: TEST_truncated_sna_record: echo 'corrupting the SnapMapper DB of osd.1 (db: td/osd-mapper/1)' 2026-02-17T11:26:46.152 INFO:tasks.workunit.client.0.trial128.stdout:corrupting the SnapMapper DB of osd.1 (db: td/osd-mapper/1) 2026-02-17T11:26:46.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: (( extr_dbg >= 3 )) 2026-02-17T11:26:46.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2026-02-17T11:26:46.165 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.154+0000 7fef642e8bc0 1 bdev(0x561cd9507800 td/osd-mapper/1/block) open path td/osd-mapper/1/block 2026-02-17T11:26:46.165 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.158+0000 7fef642e8bc0 1 bdev(0x561cd9507800 td/osd-mapper/1/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:46.166 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.158+0000 7fef642e8bc0 1 bdev(0x561cd9507800 td/osd-mapper/1/block) close 2026-02-17T11:26:46.448 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.442+0000 7fef642e8bc0 1 bdev(0x561cd9507800 td/osd-mapper/1/block) open path td/osd-mapper/1/block 2026-02-17T11:26:46.448 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.442+0000 7fef642e8bc0 1 bdev(0x561cd9507800 td/osd-mapper/1/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:46.449 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.442+0000 7fef642e8bc0 1 bluestore(td/osd-mapper/1) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T11:26:46.449 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.442+0000 7fef642e8bc0 1 bdev(0x561cd9507c00 td/osd-mapper/1/block) open path td/osd-mapper/1/block 2026-02-17T11:26:46.449 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.442+0000 7fef642e8bc0 1 bdev(0x561cd9507c00 td/osd-mapper/1/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:46.449 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.442+0000 7fef642e8bc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/1/block size 100 GiB 2026-02-17T11:26:46.449 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.442+0000 7fef642e8bc0 1 bluefs mount 2026-02-17T11:26:46.449 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.442+0000 7fef642e8bc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T11:26:46.451 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.442+0000 7fef642e8bc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:46.451 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.442+0000 7fef642e8bc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:46.451 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.442+0000 7fef642e8bc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:46.451 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.442+0000 7fef642e8bc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:46.451 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.442+0000 7fef642e8bc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:46.451 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.442+0000 7fef642e8bc0 1 bluefs mount shared_bdev_used = 0 2026-02-17T11:26:46.451 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.442+0000 7fef642e8bc0 1 bluestore(td/osd-mapper/1) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T11:26:46.474 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.466+0000 7fef642e8bc0 1 bluestore(td/osd-mapper/1) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T11:26:46.475 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.466+0000 7fef642e8bc0 1 bluestore(td/osd-mapper/1) _open_super_meta old nid_max 2051 2026-02-17T11:26:46.475 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.466+0000 7fef642e8bc0 1 bluestore(td/osd-mapper/1) _open_super_meta old blobid_max 20480 2026-02-17T11:26:46.475 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.466+0000 7fef642e8bc0 1 bluestore(td/osd-mapper/1) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T11:26:46.475 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.466+0000 7fef642e8bc0 1 bluestore(td/osd-mapper/1) _open_super_meta min_alloc_size 0x1000 2026-02-17T11:26:46.475 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.466+0000 7fef642e8bc0 1 freelist init 2026-02-17T11:26:46.476 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.466+0000 7fef642e8bc0 1 freelist _read_cfg 2026-02-17T11:26:46.477 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.470+0000 7fef642e8bc0 1 bluestore(td/osd-mapper/1) _init_alloc loaded 100 GiB in 4 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffca000, fragmentation 1.14441e-07 2026-02-17T11:26:46.478 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.470+0000 7fef642e8bc0 1 bluefs umount 2026-02-17T11:26:46.478 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.470+0000 7fef642e8bc0 1 bdev(0x561cd9507c00 td/osd-mapper/1/block) close 2026-02-17T11:26:46.729 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.722+0000 7fef642e8bc0 1 bdev(0x561cd9507c00 td/osd-mapper/1/block) open path td/osd-mapper/1/block 2026-02-17T11:26:46.729 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.722+0000 7fef642e8bc0 1 bdev(0x561cd9507c00 td/osd-mapper/1/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:46.729 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.722+0000 7fef642e8bc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/1/block size 100 GiB 2026-02-17T11:26:46.729 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.722+0000 7fef642e8bc0 1 bluefs mount 2026-02-17T11:26:46.729 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.722+0000 7fef642e8bc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T11:26:46.730 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.722+0000 7fef642e8bc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:46.730 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.722+0000 7fef642e8bc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:46.730 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.722+0000 7fef642e8bc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:46.730 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.722+0000 7fef642e8bc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:46.730 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.722+0000 7fef642e8bc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:46.731 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.722+0000 7fef642e8bc0 1 bluefs mount shared_bdev_used = 7864320 2026-02-17T11:26:46.731 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.722+0000 7fef642e8bc0 1 bluestore(td/osd-mapper/1) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T11:26:46.742 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.734+0000 7fef642e8bc0 1 bluestore(td/osd-mapper/1) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000000 0b 05 58 02 00 00 fd b2 54 7e 2a 2a 41 a0 b1 69 |..X.....T~**A..i| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000010 5d fc 68 2b 2f fb 01 00 00 00 17 00 00 00 00 00 |].h+/...........| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:* 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2026-02-17T11:26:46.743 INFO:tasks.workunit.client.0.trial128.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000220 72 65 17 00 00 00 09 00 00 00 41 36 4e f8 bd 7c |re........A6N..|| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000230 49 75 bb 7f b2 dc fd b0 f4 6e 00 00 00 00 00 00 |Iu.......n......| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000240 00 00 17 00 00 00 52 50 94 69 eb 68 4a 31 01 00 |......RP.i.hJ1..| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000250 00 00 01 00 00 00 01 00 00 00 17 00 00 00 |..............| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:0000025e 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000047 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:0000003f 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000047 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:46.744 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.734+0000 7fef642e8bc0 1 bluefs umount 2026-02-17T11:26:46.745 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:46.734+0000 7fef642e8bc0 1 bdev(0x561cd9507c00 td/osd-mapper/1/block) close 2026-02-17T11:26:47.008 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:47.002+0000 7fef642e8bc0 1 freelist shutdown 2026-02-17T11:26:47.008 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:47.002+0000 7fef642e8bc0 1 bdev(0x561cd9507800 td/osd-mapper/1/block) close 2026-02-17T11:26:47.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2026-02-17T11:26:47.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_0000000000000003_000000000000000' 2026-02-17T11:26:47.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-17T11:26:48.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: KY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:48.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T11:26:48.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: echo 'SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..' 2026-02-17T11:26:48.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: cat -v 2026-02-17T11:26:48.318 INFO:tasks.workunit.client.0.trial128.stdout:SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:48.319 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: mktemp -p /tmp --suffix=_the_val 2026-02-17T11:26:48.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: tmp_fn1=/tmp/tmp.8MqcFWttHv_the_val 2026-02-17T11:26:48.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T11:26:48.320 INFO:tasks.workunit.client.0.trial128.stdout:Value dumped in: /tmp/tmp.8MqcFWttHv_the_val 2026-02-17T11:26:48.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: echo 'Value dumped in: /tmp/tmp.8MqcFWttHv_the_val' 2026-02-17T11:26:48.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:119: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 get p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. out /tmp/tmp.8MqcFWttHv_the_val 2026-02-17T11:26:48.929 INFO:tasks.workunit.client.0.trial128.stdout:(p, %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..) 2026-02-17T11:26:49.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T11:26:49.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: od -xc /tmp/tmp.8MqcFWttHv_the_val 2026-02-17T11:26:49.414 INFO:tasks.workunit.client.0.trial128.stdout:0000000 0101 0035 0000 0003 0000 0000 0000 0304 2026-02-17T11:26:49.414 INFO:tasks.workunit.client.0.trial128.stdout: 001 001 5 \0 \0 \0 003 \0 \0 \0 \0 \0 \0 \0 004 003 2026-02-17T11:26:49.414 INFO:tasks.workunit.client.0.trial128.stdout:0000020 0027 0000 0000 0000 0006 0000 626f 786a 2026-02-17T11:26:49.414 INFO:tasks.workunit.client.0.trial128.stdout: ' \0 \0 \0 \0 \0 \0 \0 006 \0 \0 \0 o b j x 2026-02-17T11:26:49.414 INFO:tasks.workunit.client.0.trial128.stdout:0000040 7878 0003 0000 0000 0000 75af 85d0 0000 2026-02-17T11:26:49.414 INFO:tasks.workunit.client.0.trial128.stdout: x x 003 \0 \0 \0 \0 \0 \0 \0 257 u 320 205 \0 \0 2026-02-17T11:26:49.414 INFO:tasks.workunit.client.0.trial128.stdout:0000060 0000 0100 0000 0000 0000 0000 2026-02-17T11:26:49.414 INFO:tasks.workunit.client.0.trial128.stdout: \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0 2026-02-17T11:26:49.414 INFO:tasks.workunit.client.0.trial128.stdout:0000073 2026-02-17T11:26:49.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:122: TEST_truncated_sna_record: NKY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 2026-02-17T11:26:49.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:123: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 rm p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:50.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:124: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 set p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 in /tmp/tmp.8MqcFWttHv_the_val 2026-02-17T11:26:51.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:126: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T11:26:51.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:106: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T11:26:51.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:108: TEST_truncated_sna_record: kvdir=td/osd-mapper/2 2026-02-17T11:26:51.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:109: TEST_truncated_sna_record: echo 'corrupting the SnapMapper DB of osd.2 (db: td/osd-mapper/2)' 2026-02-17T11:26:51.582 INFO:tasks.workunit.client.0.trial128.stdout:corrupting the SnapMapper DB of osd.2 (db: td/osd-mapper/2) 2026-02-17T11:26:51.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: (( extr_dbg >= 3 )) 2026-02-17T11:26:51.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:110: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 dump p 2026-02-17T11:26:51.595 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.586+0000 7faff882dbc0 1 bdev(0x55eb1a41f800 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-17T11:26:51.595 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.586+0000 7faff882dbc0 1 bdev(0x55eb1a41f800 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:51.595 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.586+0000 7faff882dbc0 1 bdev(0x55eb1a41f800 td/osd-mapper/2/block) close 2026-02-17T11:26:51.876 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.870+0000 7faff882dbc0 1 bdev(0x55eb1a41f800 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-17T11:26:51.877 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.870+0000 7faff882dbc0 1 bdev(0x55eb1a41f800 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:51.877 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.870+0000 7faff882dbc0 1 bluestore(td/osd-mapper/2) _set_cache_sizes cache_size 3221225472 meta 0.45 kv 0.45 kv_onode 0.04 data 0.06 2026-02-17T11:26:51.877 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.870+0000 7faff882dbc0 1 bdev(0x55eb1a41fc00 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-17T11:26:51.877 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.870+0000 7faff882dbc0 1 bdev(0x55eb1a41fc00 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:51.877 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.870+0000 7faff882dbc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/2/block size 100 GiB 2026-02-17T11:26:51.877 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.870+0000 7faff882dbc0 1 bluefs mount 2026-02-17T11:26:51.877 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.870+0000 7faff882dbc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T11:26:51.880 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.870+0000 7faff882dbc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:51.880 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.870+0000 7faff882dbc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:51.880 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.870+0000 7faff882dbc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:51.880 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.870+0000 7faff882dbc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:51.880 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.870+0000 7faff882dbc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:51.880 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.870+0000 7faff882dbc0 1 bluefs mount shared_bdev_used = 0 2026-02-17T11:26:51.881 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.870+0000 7faff882dbc0 1 bluestore(td/osd-mapper/2) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T11:26:51.893 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.886+0000 7faff882dbc0 1 bluestore(td/osd-mapper/2) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T11:26:51.893 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.886+0000 7faff882dbc0 1 bluestore(td/osd-mapper/2) _open_super_meta old nid_max 2050 2026-02-17T11:26:51.893 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.886+0000 7faff882dbc0 1 bluestore(td/osd-mapper/2) _open_super_meta old blobid_max 20480 2026-02-17T11:26:51.893 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.886+0000 7faff882dbc0 1 bluestore(td/osd-mapper/2) _open_super_meta ondisk_format 4 compat_ondisk_format 3 2026-02-17T11:26:51.893 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.886+0000 7faff882dbc0 1 bluestore(td/osd-mapper/2) _open_super_meta min_alloc_size 0x1000 2026-02-17T11:26:51.894 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.886+0000 7faff882dbc0 1 freelist init 2026-02-17T11:26:51.894 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.886+0000 7faff882dbc0 1 freelist _read_cfg 2026-02-17T11:26:51.895 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.886+0000 7faff882dbc0 1 bluestore(td/osd-mapper/2) _init_alloc loaded 100 GiB in 5 extents, allocator type hybrid, capacity 0x1900000000, block size 0x1000, free 0x18fffca000, fragmentation 1.52588e-07 2026-02-17T11:26:51.895 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.886+0000 7faff882dbc0 1 bluefs umount 2026-02-17T11:26:51.895 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:51.886+0000 7faff882dbc0 1 bdev(0x55eb1a41fc00 td/osd-mapper/2/block) close 2026-02-17T11:26:52.164 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.158+0000 7faff882dbc0 1 bdev(0x55eb1a41fc00 td/osd-mapper/2/block) open path td/osd-mapper/2/block 2026-02-17T11:26:52.165 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.158+0000 7faff882dbc0 1 bdev(0x55eb1a41fc00 td/osd-mapper/2/block) open size 107374182400 (0x1900000000, 100 GiB) block_size 4096 (4 KiB) non-rotational device, discard supported 2026-02-17T11:26:52.165 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.158+0000 7faff882dbc0 1 bluefs add_block_device bdev 1 path td/osd-mapper/2/block size 100 GiB 2026-02-17T11:26:52.165 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.158+0000 7faff882dbc0 1 bluefs mount 2026-02-17T11:26:52.165 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.158+0000 7faff882dbc0 1 bluefs _init_alloc shared, id 1, capacity 0x1900000000, block size 0x10000 2026-02-17T11:26:52.165 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.158+0000 7faff882dbc0 1 bluefs mount final locked allocations 0 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:52.166 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.158+0000 7faff882dbc0 1 bluefs mount final locked allocations 1 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:52.166 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.158+0000 7faff882dbc0 1 bluefs mount final locked allocations 2 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:52.166 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.158+0000 7faff882dbc0 1 bluefs mount final locked allocations 3 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:52.166 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.158+0000 7faff882dbc0 1 bluefs mount final locked allocations 4 <0x0~0, [0x0~0], 0x0~0> => <0x0~0, [0x0~0], 0x0~0> 2026-02-17T11:26:52.166 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.158+0000 7faff882dbc0 1 bluefs mount shared_bdev_used = 7864320 2026-02-17T11:26:52.166 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.158+0000 7faff882dbc0 1 bluestore(td/osd-mapper/2) _prepare_db_environment set db_paths to db,102005473280 db.slow,102005473280 2026-02-17T11:26:52.184 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.178+0000 7faff882dbc0 1 bluestore(td/osd-mapper/2) _open_db opened rocksdb path db options compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0 2026-02-17T11:26:52.186 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01.osd_superblock 2026-02-17T11:26:52.186 INFO:tasks.workunit.client.0.trial128.stdout:00000000 0b 05 58 02 00 00 fd b2 54 7e 2a 2a 41 a0 b1 69 |..X.....T~**A..i| 2026-02-17T11:26:52.186 INFO:tasks.workunit.client.0.trial128.stdout:00000010 5d fc 68 2b 2f fb 02 00 00 00 17 00 00 00 00 00 |].h+/...........| 2026-02-17T11:26:52.186 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2026-02-17T11:26:52.186 INFO:tasks.workunit.client.0.trial128.stdout:* 2026-02-17T11:26:52.186 INFO:tasks.workunit.client.0.trial128.stdout:00000040 00 00 00 00 00 00 fe ff 03 00 00 00 00 00 11 00 |................| 2026-02-17T11:26:52.186 INFO:tasks.workunit.client.0.trial128.stdout:00000050 00 00 01 00 00 00 00 00 00 00 1a 00 00 00 69 6e |..............in| 2026-02-17T11:26:52.187 INFO:tasks.workunit.client.0.trial128.stdout:00000060 69 74 69 61 6c 20 66 65 61 74 75 72 65 20 73 65 |itial feature se| 2026-02-17T11:26:52.187 INFO:tasks.workunit.client.0.trial128.stdout:00000070 74 28 7e 76 2e 31 38 29 02 00 00 00 00 00 00 00 |t(~v.18)........| 2026-02-17T11:26:52.187 INFO:tasks.workunit.client.0.trial128.stdout:00000080 0d 00 00 00 70 67 69 6e 66 6f 20 6f 62 6a 65 63 |....pginfo objec| 2026-02-17T11:26:52.187 INFO:tasks.workunit.client.0.trial128.stdout:00000090 74 03 00 00 00 00 00 00 00 0e 00 00 00 6f 62 6a |t............obj| 2026-02-17T11:26:52.187 INFO:tasks.workunit.client.0.trial128.stdout:000000a0 65 63 74 20 6c 6f 63 61 74 6f 72 04 00 00 00 00 |ect locator.....| 2026-02-17T11:26:52.187 INFO:tasks.workunit.client.0.trial128.stdout:000000b0 00 00 00 10 00 00 00 6c 61 73 74 5f 65 70 6f 63 |.......last_epoc| 2026-02-17T11:26:52.187 INFO:tasks.workunit.client.0.trial128.stdout:000000c0 68 5f 63 6c 65 61 6e 05 00 00 00 00 00 00 00 0a |h_clean.........| 2026-02-17T11:26:52.187 INFO:tasks.workunit.client.0.trial128.stdout:000000d0 00 00 00 63 61 74 65 67 6f 72 69 65 73 06 00 00 |...categories...| 2026-02-17T11:26:52.187 INFO:tasks.workunit.client.0.trial128.stdout:000000e0 00 00 00 00 00 0b 00 00 00 68 6f 62 6a 65 63 74 |.........hobject| 2026-02-17T11:26:52.187 INFO:tasks.workunit.client.0.trial128.stdout:000000f0 70 6f 6f 6c 07 00 00 00 00 00 00 00 07 00 00 00 |pool............| 2026-02-17T11:26:52.188 INFO:tasks.workunit.client.0.trial128.stdout:00000100 62 69 67 69 6e 66 6f 08 00 00 00 00 00 00 00 0b |biginfo.........| 2026-02-17T11:26:52.188 INFO:tasks.workunit.client.0.trial128.stdout:00000110 00 00 00 6c 65 76 65 6c 64 62 69 6e 66 6f 09 00 |...leveldbinfo..| 2026-02-17T11:26:52.188 INFO:tasks.workunit.client.0.trial128.stdout:00000120 00 00 00 00 00 00 0a 00 00 00 6c 65 76 65 6c 64 |..........leveld| 2026-02-17T11:26:52.188 INFO:tasks.workunit.client.0.trial128.stdout:00000130 62 6c 6f 67 0a 00 00 00 00 00 00 00 0a 00 00 00 |blog............| 2026-02-17T11:26:52.188 INFO:tasks.workunit.client.0.trial128.stdout:00000140 73 6e 61 70 6d 61 70 70 65 72 0b 00 00 00 00 00 |snapmapper......| 2026-02-17T11:26:52.188 INFO:tasks.workunit.client.0.trial128.stdout:00000150 00 00 0f 00 00 00 73 68 61 72 64 65 64 20 6f 62 |......sharded ob| 2026-02-17T11:26:52.188 INFO:tasks.workunit.client.0.trial128.stdout:00000160 6a 65 63 74 73 0c 00 00 00 00 00 00 00 11 00 00 |jects...........| 2026-02-17T11:26:52.188 INFO:tasks.workunit.client.0.trial128.stdout:00000170 00 74 72 61 6e 73 61 63 74 69 6f 6e 20 68 69 6e |.transaction hin| 2026-02-17T11:26:52.188 INFO:tasks.workunit.client.0.trial128.stdout:00000180 74 73 0d 00 00 00 00 00 00 00 0e 00 00 00 70 67 |ts............pg| 2026-02-17T11:26:52.188 INFO:tasks.workunit.client.0.trial128.stdout:00000190 20 6d 65 74 61 20 6f 62 6a 65 63 74 0e 00 00 00 | meta object....| 2026-02-17T11:26:52.189 INFO:tasks.workunit.client.0.trial128.stdout:000001a0 00 00 00 00 14 00 00 00 65 78 70 6c 69 63 69 74 |........explicit| 2026-02-17T11:26:52.189 INFO:tasks.workunit.client.0.trial128.stdout:000001b0 20 6d 69 73 73 69 6e 67 20 73 65 74 0f 00 00 00 | missing set....| 2026-02-17T11:26:52.189 INFO:tasks.workunit.client.0.trial128.stdout:000001c0 00 00 00 00 10 00 00 00 66 61 73 74 69 6e 66 6f |........fastinfo| 2026-02-17T11:26:52.189 INFO:tasks.workunit.client.0.trial128.stdout:000001d0 20 70 67 20 61 74 74 72 10 00 00 00 00 00 00 00 | pg attr........| 2026-02-17T11:26:52.189 INFO:tasks.workunit.client.0.trial128.stdout:000001e0 16 00 00 00 64 65 6c 65 74 65 73 20 69 6e 20 6d |....deletes in m| 2026-02-17T11:26:52.189 INFO:tasks.workunit.client.0.trial128.stdout:000001f0 69 73 73 69 6e 67 20 73 65 74 11 00 00 00 00 00 |issing set......| 2026-02-17T11:26:52.189 INFO:tasks.workunit.client.0.trial128.stdout:00000200 00 00 1c 00 00 00 6e 65 77 20 73 6e 61 70 6d 61 |......new snapma| 2026-02-17T11:26:52.189 INFO:tasks.workunit.client.0.trial128.stdout:00000210 70 70 65 72 20 6b 65 79 20 73 74 72 75 63 74 75 |pper key structu| 2026-02-17T11:26:52.189 INFO:tasks.workunit.client.0.trial128.stdout:00000220 72 65 17 00 00 00 0d 00 00 00 ee 2c 5b 75 fc e6 |re.........,[u..| 2026-02-17T11:26:52.190 INFO:tasks.workunit.client.0.trial128.stdout:00000230 43 eb 8f a3 50 50 6a ad 42 31 00 00 00 00 00 00 |C...PPj.B1......| 2026-02-17T11:26:52.190 INFO:tasks.workunit.client.0.trial128.stdout:00000240 00 00 17 00 00 00 5b 50 94 69 6a ca 18 21 01 00 |......[P.ij..!..| 2026-02-17T11:26:52.190 INFO:tasks.workunit.client.0.trial128.stdout:00000250 00 00 01 00 00 00 01 00 00 00 17 00 00 00 |..............| 2026-02-17T11:26:52.190 INFO:tasks.workunit.client.0.trial128.stdout:0000025e 2026-02-17T11:26:52.190 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:52.190 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%7b%3fC%c4%00%00%00%00%00%00%00%01~ 2026-02-17T11:26:52.191 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:52.191 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:52.191 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T11:26:52.191 INFO:tasks.workunit.client.0.trial128.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 02 00 00 00 00 00 |....objxxx......| 2026-02-17T11:26:52.191 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T11:26:52.191 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 02 |................| 2026-02-17T11:26:52.192 INFO:tasks.workunit.client.0.trial128.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T11:26:52.192 INFO:tasks.workunit.client.0.trial128.stdout:00000047 2026-02-17T11:26:52.192 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:52.192 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:52.192 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 39 00 00 00 04 03 27 00 00 00 00 00 00 00 |..9.....'.......| 2026-02-17T11:26:52.192 INFO:tasks.workunit.client.0.trial128.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 03 00 00 00 00 00 |....objxxx......| 2026-02-17T11:26:52.192 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T11:26:52.193 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 03 00 00 00 00 00 00 00 |...............| 2026-02-17T11:26:52.193 INFO:tasks.workunit.client.0.trial128.stdout:0000003f 2026-02-17T11:26:52.193 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:52.193 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.OBJ_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:52.193 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 41 00 00 00 04 03 27 00 00 00 00 00 00 00 |..A.....'.......| 2026-02-17T11:26:52.193 INFO:tasks.workunit.client.0.trial128.stdout:00000010 06 00 00 00 6f 62 6a 78 78 78 05 00 00 00 00 00 |....objxxx......| 2026-02-17T11:26:52.194 INFO:tasks.workunit.client.0.trial128.stdout:00000020 00 00 af 75 d0 85 00 00 00 00 00 01 00 00 00 00 |...u............| 2026-02-17T11:26:52.194 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 02 00 00 00 04 00 00 00 00 00 00 00 05 |................| 2026-02-17T11:26:52.194 INFO:tasks.workunit.client.0.trial128.stdout:00000040 00 00 00 00 00 00 00 |.......| 2026-02-17T11:26:52.194 INFO:tasks.workunit.client.0.trial128.stdout:00000047 2026-02-17T11:26:52.194 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:52.194 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:52.195 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 01 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:52.195 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:52.195 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:52.195 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:52.195 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:52.195 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:52.195 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:26:52.196 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 02 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:52.196 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:52.196 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 02 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:52.196 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:52.196 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:52.196 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:52.197 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:52.197 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 03 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:52.197 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:52.197 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 03 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:52.197 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:52.197 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:52.198 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:52.198 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:52.198 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 04 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:52.198 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:52.198 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:52.198 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:52.199 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:52.199 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:52.199 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:26:52.199 INFO:tasks.workunit.client.0.trial128.stdout:00000000 01 01 35 00 00 00 05 00 00 00 00 00 00 00 04 03 |..5.............| 2026-02-17T11:26:52.199 INFO:tasks.workunit.client.0.trial128.stdout:00000010 27 00 00 00 00 00 00 00 06 00 00 00 6f 62 6a 78 |'...........objx| 2026-02-17T11:26:52.199 INFO:tasks.workunit.client.0.trial128.stdout:00000020 78 78 05 00 00 00 00 00 00 00 af 75 d0 85 00 00 |xx.........u....| 2026-02-17T11:26:52.200 INFO:tasks.workunit.client.0.trial128.stdout:00000030 00 00 00 01 00 00 00 00 00 00 00 |...........| 2026-02-17T11:26:52.200 INFO:tasks.workunit.client.0.trial128.stdout:0000003b 2026-02-17T11:26:52.200 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:52.200 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02~ 2026-02-17T11:26:52.200 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:52.200 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c1%a3%fcn%00%00%00%00%00%00%04%03~ 2026-02-17T11:26:52.201 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:26:52.201 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.178+0000 7faff882dbc0 1 bluefs umount 2026-02-17T11:26:52.202 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.178+0000 7faff882dbc0 1 bdev(0x55eb1a41fc00 td/osd-mapper/2/block) close 2026-02-17T11:26:52.444 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.438+0000 7faff882dbc0 1 freelist shutdown 2026-02-17T11:26:52.444 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:52.438+0000 7faff882dbc0 1 bdev(0x55eb1a41f800 td/osd-mapper/2/block) close 2026-02-17T11:26:52.663 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 dump p 2026-02-17T11:26:52.663 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_0000000000000003_000000000000000' 2026-02-17T11:26:52.664 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-17T11:26:53.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:114: TEST_truncated_sna_record: KY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:53.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T11:26:53.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: echo 'SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..' 2026-02-17T11:26:53.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:115: TEST_truncated_sna_record: cat -v 2026-02-17T11:26:53.751 INFO:tasks.workunit.client.0.trial128.stdout:SNA key: %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:53.752 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: mktemp -p /tmp --suffix=_the_val 2026-02-17T11:26:53.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:117: TEST_truncated_sna_record: tmp_fn1=/tmp/tmp.FDh3LBcPS1_the_val 2026-02-17T11:26:53.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T11:26:53.753 INFO:tasks.workunit.client.0.trial128.stdout:Value dumped in: /tmp/tmp.FDh3LBcPS1_the_val 2026-02-17T11:26:53.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:118: TEST_truncated_sna_record: echo 'Value dumped in: /tmp/tmp.FDh3LBcPS1_the_val' 2026-02-17T11:26:53.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:119: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 get p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. out /tmp/tmp.FDh3LBcPS1_the_val 2026-02-17T11:26:54.343 INFO:tasks.workunit.client.0.trial128.stdout:(p, %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx..) 2026-02-17T11:26:54.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T11:26:54.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:120: TEST_truncated_sna_record: od -xc /tmp/tmp.FDh3LBcPS1_the_val 2026-02-17T11:26:54.834 INFO:tasks.workunit.client.0.trial128.stdout:0000000 0101 0035 0000 0003 0000 0000 0000 0304 2026-02-17T11:26:54.834 INFO:tasks.workunit.client.0.trial128.stdout: 001 001 5 \0 \0 \0 003 \0 \0 \0 \0 \0 \0 \0 004 003 2026-02-17T11:26:54.834 INFO:tasks.workunit.client.0.trial128.stdout:0000020 0027 0000 0000 0000 0006 0000 626f 786a 2026-02-17T11:26:54.834 INFO:tasks.workunit.client.0.trial128.stdout: ' \0 \0 \0 \0 \0 \0 \0 006 \0 \0 \0 o b j x 2026-02-17T11:26:54.835 INFO:tasks.workunit.client.0.trial128.stdout:0000040 7878 0003 0000 0000 0000 75af 85d0 0000 2026-02-17T11:26:54.835 INFO:tasks.workunit.client.0.trial128.stdout: x x 003 \0 \0 \0 \0 \0 \0 \0 257 u 320 205 \0 \0 2026-02-17T11:26:54.835 INFO:tasks.workunit.client.0.trial128.stdout:0000060 0000 0100 0000 0000 0000 0000 2026-02-17T11:26:54.835 INFO:tasks.workunit.client.0.trial128.stdout: \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0 2026-02-17T11:26:54.835 INFO:tasks.workunit.client.0.trial128.stdout:0000073 2026-02-17T11:26:54.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:122: TEST_truncated_sna_record: NKY=%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 2026-02-17T11:26:54.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:123: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 rm p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:26:55.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:124: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/2 set p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 in /tmp/tmp.FDh3LBcPS1_the_val 2026-02-17T11:26:57.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:126: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T11:26:57.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:129: TEST_truncated_sna_record: orig_osd_args=' --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 ' 2026-02-17T11:26:57.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:130: TEST_truncated_sna_record: echo --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 2026-02-17T11:26:57.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:130: TEST_truncated_sna_record: orig_osd_args=' --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T11:26:57.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:131: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T11:26:57.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:131: TEST_truncated_sna_record: echo 'Copied OSD args: / --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1/ /--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1/' 2026-02-17T11:26:57.014 INFO:tasks.workunit.client.0.trial128.stdout:Copied OSD args: / --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1/ /--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1/ 2026-02-17T11:26:57.015 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: expr 3 - 1 2026-02-17T11:26:57.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: seq 0 2 2026-02-17T11:26:57.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T11:26:57.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: CEPH_ARGS='--fsid=fdb2547e-2a2a-41a0-b169-5dfc682b2ffb --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T11:26:57.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: activate_osd td/osd-mapper 0 2026-02-17T11:26:57.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-mapper 2026-02-17T11:26:57.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:26:57.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-mapper/0 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=fdb2547e-2a2a-41a0-b169-5dfc682b2ffb --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-mapper/0' 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-mapper/0/journal' 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-mapper' 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11816 2026-02-17T11:26:57.018 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.11816/$cluster-$name.asok' 2026-02-17T11:26:57.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11816/$cluster-$name.asok' 2026-02-17T11:26:57.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:26:57.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-mapper/$name.log' 2026-02-17T11:26:57.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-mapper/$name.pid' 2026-02-17T11:26:57.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:26:57.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:26:57.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:26:57.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:26:57.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:26:57.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:26:57.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-mapper/0 2026-02-17T11:26:57.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:26:57.020 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:26:57.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=fdb2547e-2a2a-41a0-b169-5dfc682b2ffb --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-mapper/0 --osd-journal=td/osd-mapper/0/journal --chdir= --run-dir=td/osd-mapper '--admin-socket=/tmp/ceph-asok.11816/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-mapper/$name.log' '--pid-file=td/osd-mapper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:26:57.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-mapper/0/whoami 2026-02-17T11:26:57.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:26:57.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:26:57.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:26:57.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:26:57.042 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:57.037+0000 7f91d56c38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:26:57.043 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:57.037+0000 7f91d56c38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:26:57.044 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:57.041+0000 7f91d56c38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:26:57.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:26:57.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:26:57.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:26:57.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:26:57.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:26:57.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:26:57.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:26:57.175 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:26:57.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:26:57.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:26:57.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:26:58.274 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:26:58.269+0000 7f91d56c38c0 -1 Falling back to public interface 2026-02-17T11:26:58.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:26:58.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:26:58.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:26:58.336 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:26:58.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:26:58.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:26:58.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:26:59.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:26:59.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:26:59.496 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:26:59.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:26:59.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:26:59.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:26:59.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:00.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:00.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:00.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:27:00.654 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:27:00.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:00.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:27:00.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:01.266 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:01.261+0000 7f91d56c38c0 -1 osd.0 23 log_to_monitors true 2026-02-17T11:27:01.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:01.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:01.815 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:27:01.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:27:01.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:01.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:27:01.971 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 26 up_thru 26 down_at 24 last_clean_interval [5,23) [v2:127.0.0.1:6802/1973423557,v1:127.0.0.1:6803/1973423557] [v2:127.0.0.1:6804/1973423557,v1:127.0.0.1:6805/1973423557] exists,up b706816b-c6ef-45c0-a473-57c73761fb52 2026-02-17T11:27:01.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:27:01.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:27:01.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:27:01.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T11:27:01.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: CEPH_ARGS='--fsid=fdb2547e-2a2a-41a0-b169-5dfc682b2ffb --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T11:27:01.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: activate_osd td/osd-mapper 1 2026-02-17T11:27:01.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-mapper 2026-02-17T11:27:01.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:27:01.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:27:01.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:27:01.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-mapper/1 2026-02-17T11:27:01.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=fdb2547e-2a2a-41a0-b169-5dfc682b2ffb --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T11:27:01.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:27:01.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:27:01.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:27:01.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-mapper/1' 2026-02-17T11:27:01.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-mapper/1/journal' 2026-02-17T11:27:01.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:27:01.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:27:01.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-mapper' 2026-02-17T11:27:01.974 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:27:01.974 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:27:01.974 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:27:01.974 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:27:01.974 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:27:01.974 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11816 2026-02-17T11:27:01.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.11816/$cluster-$name.asok' 2026-02-17T11:27:01.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11816/$cluster-$name.asok' 2026-02-17T11:27:01.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:27:01.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-mapper/$name.log' 2026-02-17T11:27:01.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-mapper/$name.pid' 2026-02-17T11:27:01.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:27:01.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:27:01.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:27:01.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:27:01.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:27:01.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:27:01.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-mapper/1 2026-02-17T11:27:01.976 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:27:01.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:27:01.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=fdb2547e-2a2a-41a0-b169-5dfc682b2ffb --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-mapper/1 --osd-journal=td/osd-mapper/1/journal --chdir= --run-dir=td/osd-mapper '--admin-socket=/tmp/ceph-asok.11816/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-mapper/$name.log' '--pid-file=td/osd-mapper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:27:01.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-mapper/1/whoami 2026-02-17T11:27:01.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:27:01.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:27:01.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:27:01.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:27:02.001 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:01.997+0000 7f78ab77e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:02.002 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:01.997+0000 7f78ab77e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:02.003 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:01.997+0000 7f78ab77e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:02.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:27:02.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:27:02.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:27:02.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:27:02.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:27:02.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:02.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:27:02.136 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:27:02.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:02.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:27:02.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:03.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:03.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:03.294 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:27:03.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:27:03.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:03.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:27:03.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:03.458 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:03.453+0000 7f78ab77e8c0 -1 Falling back to public interface 2026-02-17T11:27:04.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:04.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:04.456 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:27:04.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:27:04.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:04.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:27:04.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:05.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:05.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:05.618 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:27:05.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:27:05.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:05.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:27:05.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:06.447 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:06.441+0000 7f78ab77e8c0 -1 osd.1 23 log_to_monitors true 2026-02-17T11:27:06.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:06.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:06.777 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:27:06.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:27:06.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:06.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:27:06.935 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 29 up_thru 29 down_at 24 last_clean_interval [9,23) [v2:127.0.0.1:6810/2970709441,v1:127.0.0.1:6811/2970709441] [v2:127.0.0.1:6812/2970709441,v1:127.0.0.1:6813/2970709441] exists,up 41364ef8-bd7c-4975-bb7f-b2dcfdb0f46e 2026-02-17T11:27:06.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:27:06.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:27:06.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:27:06.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:132: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T11:27:06.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: CEPH_ARGS='--fsid=fdb2547e-2a2a-41a0-b169-5dfc682b2ffb --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T11:27:06.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:134: TEST_truncated_sna_record: activate_osd td/osd-mapper 2 2026-02-17T11:27:06.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-mapper 2026-02-17T11:27:06.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:27:06.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T11:27:06.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:27:06.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-mapper/2 2026-02-17T11:27:06.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=fdb2547e-2a2a-41a0-b169-5dfc682b2ffb --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T11:27:06.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:27:06.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:27:06.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:27:06.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-mapper/2' 2026-02-17T11:27:06.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-mapper/2/journal' 2026-02-17T11:27:06.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:27:06.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:27:06.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-mapper' 2026-02-17T11:27:06.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:27:06.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:27:06.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:27:06.941 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:27:06.941 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:27:06.942 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11816 2026-02-17T11:27:06.942 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.11816/$cluster-$name.asok' 2026-02-17T11:27:06.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11816/$cluster-$name.asok' 2026-02-17T11:27:06.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:27:06.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-mapper/$name.log' 2026-02-17T11:27:06.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-mapper/$name.pid' 2026-02-17T11:27:06.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:27:06.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:27:06.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:27:06.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:27:06.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:27:06.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:27:06.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-mapper/2 2026-02-17T11:27:06.943 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T11:27:06.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T11:27:06.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=fdb2547e-2a2a-41a0-b169-5dfc682b2ffb --auth-supported=none --mon-host=127.0.0.1:7144 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_pool_default_pg_autoscale_mode=off --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-mapper/2 --osd-journal=td/osd-mapper/2/journal --chdir= --run-dir=td/osd-mapper '--admin-socket=/tmp/ceph-asok.11816/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-mapper/$name.log' '--pid-file=td/osd-mapper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:27:06.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-mapper/2/whoami 2026-02-17T11:27:06.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T11:27:06.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:27:06.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:27:06.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:27:06.959 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:06.953+0000 7f3bc391b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:06.961 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:06.957+0000 7f3bc391b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:06.961 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:06.957+0000 7f3bc391b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:07.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T11:27:07.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:27:07.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:27:07.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:27:07.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:27:07.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:07.102 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:27:07.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:27:07.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:07.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:27:07.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:08.190 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:08.185+0000 7f3bc391b8c0 -1 Falling back to public interface 2026-02-17T11:27:08.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:08.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:08.257 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:27:08.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:27:08.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:08.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:27:08.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:09.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:09.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:09.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:27:09.419 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:27:09.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:09.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:27:09.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:10.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:10.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:10.577 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:27:10.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:27:10.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:10.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:27:10.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:11.472 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:11.465+0000 7f3bc391b8c0 -1 osd.2 23 log_to_monitors true 2026-02-17T11:27:11.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:11.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:11.736 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:27:11.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:27:11.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:11.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:27:11.889 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 33 up_thru 33 down_at 24 last_clean_interval [13,23) [v2:127.0.0.1:6818/2548425586,v1:127.0.0.1:6819/2548425586] [v2:127.0.0.1:6820/2548425586,v1:127.0.0.1:6821/2548425586] exists,up ee2c5b75-fce6-43eb-8fa3-50506aad4231 2026-02-17T11:27:11.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:27:11.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:27:11.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:27:11.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:136: TEST_truncated_sna_record: sleep 1 2026-02-17T11:27:12.892 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: expr 3 - 1 2026-02-17T11:27:12.893 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: seq 0 2 2026-02-17T11:27:12.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T11:27:12.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:140: TEST_truncated_sna_record: timeout 60 ceph tell osd.0 version 2026-02-17T11:27:12.960 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T11:27:12.960 INFO:tasks.workunit.client.0.trial128.stdout: "version": "19.2.3-675-g3c96dfb3", 2026-02-17T11:27:12.960 INFO:tasks.workunit.client.0.trial128.stdout: "release": "squid", 2026-02-17T11:27:12.960 INFO:tasks.workunit.client.0.trial128.stdout: "release_type": "stable" 2026-02-17T11:27:12.960 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:27:12.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T11:27:12.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:140: TEST_truncated_sna_record: timeout 60 ceph tell osd.1 version 2026-02-17T11:27:13.031 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T11:27:13.031 INFO:tasks.workunit.client.0.trial128.stdout: "version": "19.2.3-675-g3c96dfb3", 2026-02-17T11:27:13.032 INFO:tasks.workunit.client.0.trial128.stdout: "release": "squid", 2026-02-17T11:27:13.032 INFO:tasks.workunit.client.0.trial128.stdout: "release_type": "stable" 2026-02-17T11:27:13.032 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:27:13.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:138: TEST_truncated_sna_record: for sdn in $(seq 0 $(expr $osdn - 1)) 2026-02-17T11:27:13.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:140: TEST_truncated_sna_record: timeout 60 ceph tell osd.2 version 2026-02-17T11:27:13.104 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T11:27:13.104 INFO:tasks.workunit.client.0.trial128.stdout: "version": "19.2.3-675-g3c96dfb3", 2026-02-17T11:27:13.104 INFO:tasks.workunit.client.0.trial128.stdout: "release": "squid", 2026-02-17T11:27:13.104 INFO:tasks.workunit.client.0.trial128.stdout: "release_type": "stable" 2026-02-17T11:27:13.104 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:27:13.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:142: TEST_truncated_sna_record: rados --format json-pretty -p test listsnaps objxxx 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: "name": "objxxx", 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 5, 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: "id": 2, 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: "snapshots": [ 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: "id": 1, 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: "name": "snap01" 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: "id": 2, 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: "name": "snap02" 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: "size": 6, 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: "overlaps": [] 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: "id": 3, 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: "snapshots": [ 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: "id": 3, 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: "name": "snap13" 2026-02-17T11:27:13.130 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: "size": 6, 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: "overlaps": [] 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: "id": 5, 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: "snapshots": [ 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: "id": 4, 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: "name": "snap24" 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: "id": 5, 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: "name": "snap25" 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: "size": 6, 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: "overlaps": [] 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: "id": "head", 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: "snapshots": [], 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: "size": 5 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:27:13.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:145: TEST_truncated_sna_record: ceph osd unset nodeep-scrub 2026-02-17T11:27:13.335 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is unset 2026-02-17T11:27:13.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:146: TEST_truncated_sna_record: ceph osd unset noscrub 2026-02-17T11:27:13.538 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is unset 2026-02-17T11:27:13.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:149: TEST_truncated_sna_record: ceph --format=json-pretty osd map test objxxx 2026-02-17T11:27:13.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:149: TEST_truncated_sna_record: jq -r '.up[0]' 2026-02-17T11:27:13.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:149: TEST_truncated_sna_record: local cur_prim=1 2026-02-17T11:27:13.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:150: TEST_truncated_sna_record: ceph pg dump pgs 2026-02-17T11:27:13.848 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T11:27:13.848 INFO:tasks.workunit.client.0.trial128.stdout:1.3 4 0 4 0 0 23 0 0 7 0 7 active+undersized+degraded 2026-02-17T11:27:06.671959+0000 21'7 31:50 [1,0] 1 [1,0] 1 21'7 2026-02-17T11:26:26.794934+0000 21'7 2026-02-17T11:26:26.794934+0000 0 1 periodic scrub scheduled @ 2026-02-18T11:26:26.794934+0000 4 0 2026-02-17T11:27:13.848 INFO:tasks.workunit.client.0.trial128.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-17T11:27:07.434118+0000 0'0 31:25 [0,1] 0 [0,1] 0 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 no scrub is scheduled 0 0 2026-02-17T11:27:13.848 INFO:tasks.workunit.client.0.trial128.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-17T11:27:07.434140+0000 0'0 31:17 [0,1] 0 [0,1] 0 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 no scrub is scheduled 0 0 2026-02-17T11:27:13.848 INFO:tasks.workunit.client.0.trial128.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-17T11:27:06.674895+0000 0'0 31:19 [1,0] 1 [1,0] 1 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:26:20.315961+0000 0 0 2026-02-17T11:27:13.848 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:27:13.848 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T11:27:13.849 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:27:13.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:151: TEST_truncated_sna_record: sleep 2 2026-02-17T11:27:15.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:152: TEST_truncated_sna_record: ceph pg 1.3 deep-scrub 2026-02-17T11:27:15.923 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T11:27:15.923 INFO:tasks.workunit.client.0.trial128.stdout: "deep": true, 2026-02-17T11:27:15.923 INFO:tasks.workunit.client.0.trial128.stdout: "must": true, 2026-02-17T11:27:15.923 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "0.000000" 2026-02-17T11:27:15.923 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:27:15.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:153: TEST_truncated_sna_record: sleep 5 2026-02-17T11:27:20.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:154: TEST_truncated_sna_record: ceph pg dump pgs 2026-02-17T11:27:21.069 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T11:27:21.069 INFO:tasks.workunit.client.0.trial128.stdout:1.3 4 0 0 0 0 23 0 0 7 0 7 active+clean 2026-02-17T11:27:16.366087+0000 21'7 40:80 [1,2,0] 1 [1,2,0] 1 21'7 2026-02-17T11:27:16.366020+0000 21'7 2026-02-17T11:27:16.366020+0000 0 1 periodic scrub scheduled @ 2026-02-18T11:27:16.366020+0000 4 0 2026-02-17T11:27:21.069 INFO:tasks.workunit.client.0.trial128.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T11:27:11.833703+0000 0'0 40:46 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:26:20.315961+0000 0 0 2026-02-17T11:27:21.069 INFO:tasks.workunit.client.0.trial128.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T11:27:11.733032+0000 0'0 40:31 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:26:20.315961+0000 0 0 2026-02-17T11:27:21.069 INFO:tasks.workunit.client.0.trial128.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T11:27:11.833547+0000 0'0 40:39 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:26:20.315961+0000 0 0 2026-02-17T11:27:21.069 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:27:21.069 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T11:27:21.069 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:27:21.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:155: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T11:27:21.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:155: TEST_truncated_sna_record: grep -a ERR td/osd-mapper/osd.1.log 2026-02-17T11:27:21.078 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:27:16.361+0000 7f788bf01640 -1 log_channel(cluster) log [ERR] : osd.1 found snap mapper error on pg 1.3 oid 1:f5ae0ba1:::objxxx:3 snaps in mapper: {}, oi: {3} ...repaired 2026-02-17T11:27:21.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:156: TEST_truncated_sna_record: grep -a -q ERR td/osd-mapper/osd.1.log 2026-02-17T11:27:21.081 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:160: TEST_truncated_sna_record: grep -a ERR td/osd-mapper/osd.1.log 2026-02-17T11:27:21.081 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:160: TEST_truncated_sna_record: wc -l 2026-02-17T11:27:21.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:160: TEST_truncated_sna_record: local prev_err_cnt=1 2026-02-17T11:27:21.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:161: TEST_truncated_sna_record: echo 'prev count: 1' 2026-02-17T11:27:21.084 INFO:tasks.workunit.client.0.trial128.stdout:prev count: 1 2026-02-17T11:27:21.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:164: TEST_truncated_sna_record: ceph pg 1.3 deep-scrub 2026-02-17T11:27:21.151 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T11:27:21.152 INFO:tasks.workunit.client.0.trial128.stdout: "deep": true, 2026-02-17T11:27:21.152 INFO:tasks.workunit.client.0.trial128.stdout: "must": true, 2026-02-17T11:27:21.152 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "0.000000" 2026-02-17T11:27:21.152 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:27:21.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:165: TEST_truncated_sna_record: sleep 5 2026-02-17T11:27:26.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:166: TEST_truncated_sna_record: ceph pg dump pgs 2026-02-17T11:27:26.302 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T11:27:26.302 INFO:tasks.workunit.client.0.trial128.stdout:1.3 4 0 0 0 0 23 0 0 7 0 7 active+clean 2026-02-17T11:27:21.315726+0000 21'7 40:89 [1,2,0] 1 [1,2,0] 1 21'7 2026-02-17T11:27:21.315644+0000 21'7 2026-02-17T11:27:21.315644+0000 0 1 periodic scrub scheduled @ 2026-02-18T11:27:21.315644+0000 4 0 2026-02-17T11:27:26.302 INFO:tasks.workunit.client.0.trial128.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T11:27:11.833703+0000 0'0 40:46 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:26:20.315961+0000 0 0 2026-02-17T11:27:26.302 INFO:tasks.workunit.client.0.trial128.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T11:27:11.733032+0000 0'0 40:31 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:26:20.315961+0000 0 0 2026-02-17T11:27:26.302 INFO:tasks.workunit.client.0.trial128.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T11:27:11.833547+0000 0'0 40:39 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-17T11:26:20.315961+0000 0'0 2026-02-17T11:26:20.315961+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:26:20.315961+0000 0 0 2026-02-17T11:27:26.302 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:27:26.303 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T11:27:26.303 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:27:26.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:167: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T11:27:26.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:167: TEST_truncated_sna_record: grep -a ERR td/osd-mapper/osd.1.log 2026-02-17T11:27:26.312 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:27:16.361+0000 7f788bf01640 -1 log_channel(cluster) log [ERR] : osd.1 found snap mapper error on pg 1.3 oid 1:f5ae0ba1:::objxxx:3 snaps in mapper: {}, oi: {3} ...repaired 2026-02-17T11:27:26.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:168: TEST_truncated_sna_record: grep -a ERR td/osd-mapper/osd.1.log 2026-02-17T11:27:26.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:168: TEST_truncated_sna_record: wc -l 2026-02-17T11:27:26.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:168: TEST_truncated_sna_record: local current_err_cnt=1 2026-02-17T11:27:26.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:169: TEST_truncated_sna_record: (( extr_dbg >= 1 )) 2026-02-17T11:27:26.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:169: TEST_truncated_sna_record: echo 'current count: 1' 2026-02-17T11:27:26.316 INFO:tasks.workunit.client.0.trial128.stdout:current count: 1 2026-02-17T11:27:26.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:170: TEST_truncated_sna_record: (( current_err_cnt == prev_err_cnt )) 2026-02-17T11:27:26.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:171: TEST_truncated_sna_record: kill_daemons td/osd-mapper TERM osd 2026-02-17T11:27:26.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:27:26.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:27:26.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:27:26.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:27:26.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:27:27.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:27:27.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:172: TEST_truncated_sna_record: kvdir=td/osd-mapper/1 2026-02-17T11:27:27.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:173: TEST_truncated_sna_record: (( extr_dbg >= 2 )) 2026-02-17T11:27:27.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:173: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2026-02-17T11:27:27.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:173: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_' 2026-02-17T11:27:27.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:174: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-17T11:27:28.705 INFO:tasks.workunit.client.0.trial128.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:27:28.705 INFO:tasks.workunit.client.0.trial128.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.FA570D58.2.objxxx.. 2026-02-17T11:27:28.705 INFO:tasks.workunit.client.0.trial128.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_000000 2026-02-17T11:27:28.705 INFO:tasks.workunit.client.0.trial128.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.FA570D58.3.objxxx.. 2026-02-17T11:27:28.705 INFO:tasks.workunit.client.0.trial128.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:27:28.706 INFO:tasks.workunit.client.0.trial128.stdout:%00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.FA570D58.5.objxxx.. 2026-02-17T11:27:28.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: ceph-kvstore-tool bluestore-kv td/osd-mapper/1 dump p 2026-02-17T11:27:28.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: grep -a -e 'SNA_[0-9]_000000000000000[0-9]_000000000000000' 2026-02-17T11:27:28.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: awk -e '{print $2;}' 2026-02-17T11:27:28.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: wc -l 2026-02-17T11:27:29.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:176: TEST_truncated_sna_record: local num_sna_full=5 2026-02-17T11:27:29.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:177: TEST_truncated_sna_record: (( num_sna_full == num_sna_b4 )) 2026-02-17T11:27:29.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:178: TEST_truncated_sna_record: return 0 2026-02-17T11:27:29.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-mapper.sh:24: run: teardown td/osd-mapper 2026-02-17T11:27:29.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-mapper 2026-02-17T11:27:29.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:27:29.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-mapper KILL 2026-02-17T11:27:29.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:27:29.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:27:29.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:27:29.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:27:29.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:27:29.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:27:29.903 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:27:29.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:27:29.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:27:29.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:27:29.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:27:29.906 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:27:29.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:27:29.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:27:29.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:27:29.908 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:27:29.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:27:29.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:27:29.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-mapper 2026-02-17T11:27:29.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:27:29.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:27:29.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11816 2026-02-17T11:27:29.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.11816 2026-02-17T11:27:29.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:27:29.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:27:29.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T11:27:29.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-mapper 0 2026-02-17T11:27:29.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-mapper 2026-02-17T11:27:29.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T11:27:29.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-mapper KILL 2026-02-17T11:27:29.925 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:27:29.925 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:27:29.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:27:29.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:27:29.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:27:29.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:27:29.928 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:27:29.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:27:29.929 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:27:29.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:27:29.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:27:29.931 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:27:29.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:27:29.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:27:29.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:27:29.933 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:27:29.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:27:29.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T11:27:29.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-mapper 2026-02-17T11:27:29.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:27:29.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:27:29.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.11816 2026-02-17T11:27:29.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.11816 2026-02-17T11:27:29.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:27:29.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:27:29.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T11:27:29.938 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T11:27:29.938 DEBUG:teuthology.orchestra.run.trial128:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T11:27:29.989 INFO:tasks.workunit:Running workunit scrub/osd-recovery-scrub.sh... 2026-02-17T11:27:29.989 DEBUG:teuthology.orchestra.run.trial128:workunit test scrub/osd-recovery-scrub.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3c96dfb3f50b3d02598a82af7104764bfdcd3919 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh 2026-02-17T11:27:30.042 INFO:tasks.workunit.client.0.trial128.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T11:27:30.045 INFO:tasks.workunit.client.0.trial128.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T11:27:30.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T11:27:30.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T11:27:30.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T11:27:30.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T11:27:30.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T11:27:30.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T11:27:30.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T11:27:30.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T11:27:30.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-recovery-scrub 2026-02-17T11:27:30.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:20: run: local dir=td/osd-recovery-scrub 2026-02-17T11:27:30.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:21: run: shift 2026-02-17T11:27:30.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:23: run: export CEPH_MON=127.0.0.1:7124 2026-02-17T11:27:30.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:23: run: CEPH_MON=127.0.0.1:7124 2026-02-17T11:27:30.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:24: run: export CEPH_ARGS 2026-02-17T11:27:30.046 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:25: run: uuidgen 2026-02-17T11:27:30.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:25: run: CEPH_ARGS+='--fsid=14e8264a-2160-44dd-98ca-5157ccfa6ac4 --auth-supported=none ' 2026-02-17T11:27:30.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:26: run: CEPH_ARGS+='--mon-host=127.0.0.1:7124 ' 2026-02-17T11:27:30.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:27: run: CEPH_ARGS+='--osd-op-queue=wpq ' 2026-02-17T11:27:30.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:29: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-17T11:27:30.047 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:30: run: set 2026-02-17T11:27:30.047 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:30: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T11:27:30.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:30: run: local funcs=TEST_recovery_scrub_1 2026-02-17T11:27:30.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:31: run: for func in $funcs 2026-02-17T11:27:30.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:32: run: TEST_recovery_scrub_1 td/osd-recovery-scrub 2026-02-17T11:27:30.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:39: TEST_recovery_scrub_1: local dir=td/osd-recovery-scrub 2026-02-17T11:27:30.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:40: TEST_recovery_scrub_1: local poolname=test 2026-02-17T11:27:30.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:42: TEST_recovery_scrub_1: TESTDATA=testdata.31716 2026-02-17T11:27:30.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:43: TEST_recovery_scrub_1: OSDS=4 2026-02-17T11:27:30.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:44: TEST_recovery_scrub_1: PGS=1 2026-02-17T11:27:30.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:45: TEST_recovery_scrub_1: OBJECTS=100 2026-02-17T11:27:30.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:46: TEST_recovery_scrub_1: ERRORS=0 2026-02-17T11:27:30.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:48: TEST_recovery_scrub_1: setup td/osd-recovery-scrub 2026-02-17T11:27:30.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-recovery-scrub 2026-02-17T11:27:30.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-recovery-scrub 2026-02-17T11:27:30.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-recovery-scrub 2026-02-17T11:27:30.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:27:30.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-recovery-scrub KILL 2026-02-17T11:27:30.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:27:30.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:27:30.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:27:30.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:27:30.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:27:30.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:27:30.052 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:27:30.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:27:30.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:27:30.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:27:30.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:27:30.054 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:27:30.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:27:30.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:27:30.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:27:30.056 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:27:30.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:27:30.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:27:30.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-recovery-scrub 2026-02-17T11:27:30.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:27:30.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:27:30.060 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31716 2026-02-17T11:27:30.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.31716 2026-02-17T11:27:30.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:27:30.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:27:30.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-recovery-scrub 2026-02-17T11:27:30.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:27:30.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:27:30.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31716 2026-02-17T11:27:30.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.31716 2026-02-17T11:27:30.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:27:30.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T11:27:30.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T11:27:30.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:27:30.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-recovery-scrub 1' TERM HUP INT 2026-02-17T11:27:30.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:49: TEST_recovery_scrub_1: run_mon td/osd-recovery-scrub a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T11:27:30.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-recovery-scrub 2026-02-17T11:27:30.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T11:27:30.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T11:27:30.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T11:27:30.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-recovery-scrub/a 2026-02-17T11:27:30.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-recovery-scrub/a --run-dir=td/osd-recovery-scrub --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T11:27:30.089 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T11:27:30.089 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:27:30.090 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:27:30.090 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:27:30.091 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:27:30.091 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31716 2026-02-17T11:27:30.091 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.31716/$cluster-$name.asok' 2026-02-17T11:27:30.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-recovery-scrub/a '--log-file=td/osd-recovery-scrub/$name.log' '--admin-socket=/tmp/ceph-asok.31716/$cluster-$name.asok' --mon-cluster-log-file=td/osd-recovery-scrub/log --run-dir=td/osd-recovery-scrub '--pid-file=td/osd-recovery-scrub/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T11:27:30.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T11:27:30.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T11:27:30.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:27:30.127 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:27:30.127 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T11:27:30.127 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T11:27:30.127 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:27:30.127 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:27:30.127 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:27:30.127 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:27:30.127 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:27:30.128 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31716 2026-02-17T11:27:30.128 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.31716/ceph-mon.a.asok 2026-02-17T11:27:30.128 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:27:30.128 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.31716/ceph-mon.a.asok config get fsid 2026-02-17T11:27:30.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T11:27:30.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:27:30.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:27:30.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T11:27:30.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T11:27:30.183 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:27:30.183 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:27:30.183 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:27:30.183 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:27:30.183 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:27:30.183 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31716 2026-02-17T11:27:30.183 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.31716/ceph-mon.a.asok 2026-02-17T11:27:30.184 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:27:30.184 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.31716/ceph-mon.a.asok config get mon_host 2026-02-17T11:27:30.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:50: TEST_recovery_scrub_1: run_mgr td/osd-recovery-scrub x 2026-02-17T11:27:30.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-recovery-scrub 2026-02-17T11:27:30.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T11:27:30.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T11:27:30.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T11:27:30.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-recovery-scrub/x 2026-02-17T11:27:30.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T11:27:30.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T11:27:30.329 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:27:30.329 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:27:30.329 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:27:30.329 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:27:30.329 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31716 2026-02-17T11:27:30.330 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.31716/$cluster-$name.asok' 2026-02-17T11:27:30.330 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:27:30.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-recovery-scrub/x '--log-file=td/osd-recovery-scrub/$name.log' '--admin-socket=/tmp/ceph-asok.31716/$cluster-$name.asok' --run-dir=td/osd-recovery-scrub '--pid-file=td/osd-recovery-scrub/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:27:30.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:51: TEST_recovery_scrub_1: local 'ceph_osd_args=--osd-scrub-interval-randomize-ratio=0 ' 2026-02-17T11:27:30.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:52: TEST_recovery_scrub_1: ceph_osd_args+='--osd_scrub_backoff_ratio=0 ' 2026-02-17T11:27:30.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:53: TEST_recovery_scrub_1: ceph_osd_args+='--osd_stats_update_period_not_scrubbing=3 ' 2026-02-17T11:27:30.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:54: TEST_recovery_scrub_1: ceph_osd_args+=--osd_stats_update_period_scrubbing=2 2026-02-17T11:27:30.350 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: expr 4 - 1 2026-02-17T11:27:30.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: seq 0 3 2026-02-17T11:27:30.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T11:27:30.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:57: TEST_recovery_scrub_1: run_osd td/osd-recovery-scrub 0 --osd_scrub_during_recovery=false 2026-02-17T11:27:30.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2026-02-17T11:27:30.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:27:30.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T11:27:30.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:27:30.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/0 2026-02-17T11:27:30.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=14e8264a-2160-44dd-98ca-5157ccfa6ac4 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2026-02-17T11:27:30.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:27:30.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:27:30.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:27:30.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/0' 2026-02-17T11:27:30.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/0/journal' 2026-02-17T11:27:30.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:27:30.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:27:30.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2026-02-17T11:27:30.361 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:27:30.361 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:27:30.361 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:27:30.361 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:27:30.361 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:27:30.361 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31716 2026-02-17T11:27:30.362 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.31716/$cluster-$name.asok' 2026-02-17T11:27:30.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31716/$cluster-$name.asok' 2026-02-17T11:27:30.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:27:30.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:27:30.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:27:30.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2026-02-17T11:27:30.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2026-02-17T11:27:30.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:27:30.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:27:30.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:27:30.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:27:30.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:27:30.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_scrub_during_recovery=false 2026-02-17T11:27:30.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/0 2026-02-17T11:27:30.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:27:30.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=d01d043a-c7f3-4aeb-a634-c36a9544478c 2026-02-17T11:27:30.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 d01d043a-c7f3-4aeb-a634-c36a9544478c' 2026-02-17T11:27:30.363 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 d01d043a-c7f3-4aeb-a634-c36a9544478c 2026-02-17T11:27:30.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:27:30.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCiUJRpJ1+LFRAA9cn5AwMlAUtQu8ero0SCSA== 2026-02-17T11:27:30.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCiUJRpJ1+LFRAA9cn5AwMlAUtQu8ero0SCSA=="}' 2026-02-17T11:27:30.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new d01d043a-c7f3-4aeb-a634-c36a9544478c -i td/osd-recovery-scrub/0/new.json 2026-02-17T11:27:30.456 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:27:30.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/0/new.json 2026-02-17T11:27:30.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=14e8264a-2160-44dd-98ca-5157ccfa6ac4 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/0 --osd-journal=td/osd-recovery-scrub/0/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.31716/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --mkfs --key AQCiUJRpJ1+LFRAA9cn5AwMlAUtQu8ero0SCSA== --osd-uuid d01d043a-c7f3-4aeb-a634-c36a9544478c 2026-02-17T11:27:30.480 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:30.472+0000 7f18f60028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:30.481 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:30.476+0000 7f18f60028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:30.483 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:30.476+0000 7f18f60028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:30.483 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:30.476+0000 7f18f60028c0 -1 bdev(0x555928cadc00 td/osd-recovery-scrub/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:27:30.483 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:30.476+0000 7f18f60028c0 -1 bluestore(td/osd-recovery-scrub/0) _read_fsid unparsable uuid 2026-02-17T11:27:33.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/0/keyring 2026-02-17T11:27:33.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:27:33.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T11:27:33.280 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:27:33.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:27:33.470 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:27:33.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T11:27:33.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=14e8264a-2160-44dd-98ca-5157ccfa6ac4 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/0 --osd-journal=td/osd-recovery-scrub/0/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.31716/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false 2026-02-17T11:27:33.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:27:33.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:27:33.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:27:33.492 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:33.484+0000 7fca254198c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:33.494 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:33.488+0000 7fca254198c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:33.495 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:33.488+0000 7fca254198c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:33.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T11:27:33.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:27:33.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:27:33.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:27:33.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:27:33.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:33.629 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:27:33.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:27:33.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:33.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:27:33.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:34.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:34.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:34.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:27:34.779 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:27:34.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:34.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:27:34.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:34.995 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:34.988+0000 7fca254198c0 -1 Falling back to public interface 2026-02-17T11:27:35.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:35.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:35.929 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:27:35.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:27:35.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:35.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:27:36.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:37.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:37.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:37.079 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:27:37.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:27:37.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:37.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:27:37.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:38.005 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:37.996+0000 7fca254198c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:27:38.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:38.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:38.236 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:27:38.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:27:38.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:38.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:27:38.382 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1599222983,v1:127.0.0.1:6803/1599222983] [v2:127.0.0.1:6804/1599222983,v1:127.0.0.1:6805/1599222983] exists,up d01d043a-c7f3-4aeb-a634-c36a9544478c 2026-02-17T11:27:38.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:27:38.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:27:38.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:27:38.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T11:27:38.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:57: TEST_recovery_scrub_1: run_osd td/osd-recovery-scrub 1 --osd_scrub_during_recovery=false 2026-02-17T11:27:38.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2026-02-17T11:27:38.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:27:38.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T11:27:38.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:27:38.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/1 2026-02-17T11:27:38.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=14e8264a-2160-44dd-98ca-5157ccfa6ac4 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2026-02-17T11:27:38.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:27:38.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:27:38.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:27:38.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/1' 2026-02-17T11:27:38.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/1/journal' 2026-02-17T11:27:38.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:27:38.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:27:38.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2026-02-17T11:27:38.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:27:38.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:27:38.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31716 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.31716/$cluster-$name.asok' 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31716/$cluster-$name.asok' 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_scrub_during_recovery=false 2026-02-17T11:27:38.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/1 2026-02-17T11:27:38.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:27:38.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=5b079d49-2751-48ad-83e1-9c407704f078 2026-02-17T11:27:38.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 5b079d49-2751-48ad-83e1-9c407704f078' 2026-02-17T11:27:38.387 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 5b079d49-2751-48ad-83e1-9c407704f078 2026-02-17T11:27:38.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:27:38.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCqUJRpotGLFxAA0ZLKG/Xinf6mqXEUT/eZ/A== 2026-02-17T11:27:38.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCqUJRpotGLFxAA0ZLKG/Xinf6mqXEUT/eZ/A=="}' 2026-02-17T11:27:38.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 5b079d49-2751-48ad-83e1-9c407704f078 -i td/osd-recovery-scrub/1/new.json 2026-02-17T11:27:38.549 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:27:38.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/1/new.json 2026-02-17T11:27:38.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=14e8264a-2160-44dd-98ca-5157ccfa6ac4 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/1 --osd-journal=td/osd-recovery-scrub/1/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.31716/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --mkfs --key AQCqUJRpotGLFxAA0ZLKG/Xinf6mqXEUT/eZ/A== --osd-uuid 5b079d49-2751-48ad-83e1-9c407704f078 2026-02-17T11:27:38.574 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:38.568+0000 7fefac1558c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:38.575 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:38.568+0000 7fefac1558c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:38.576 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:38.568+0000 7fefac1558c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:38.577 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:38.568+0000 7fefac1558c0 -1 bdev(0x5649d61e5c00 td/osd-recovery-scrub/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:27:38.577 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:38.572+0000 7fefac1558c0 -1 bluestore(td/osd-recovery-scrub/1) _read_fsid unparsable uuid 2026-02-17T11:27:40.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/1/keyring 2026-02-17T11:27:40.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:27:40.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T11:27:40.844 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:27:40.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:27:41.042 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:27:41.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T11:27:41.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=14e8264a-2160-44dd-98ca-5157ccfa6ac4 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/1 --osd-journal=td/osd-recovery-scrub/1/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.31716/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false 2026-02-17T11:27:41.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:27:41.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:27:41.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:27:41.058 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:41.052+0000 7efcb15b78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:41.059 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:41.052+0000 7efcb15b78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:41.060 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:41.052+0000 7efcb15b78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:41.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T11:27:41.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:27:41.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:27:41.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:27:41.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:27:41.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:41.197 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:27:41.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:27:41.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:41.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:27:41.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:42.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:42.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:42.346 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:27:42.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:27:42.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:42.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:27:42.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:42.563 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:42.556+0000 7efcb15b78c0 -1 Falling back to public interface 2026-02-17T11:27:43.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:43.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:43.494 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:27:43.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:27:43.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:43.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:27:43.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:44.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:44.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:44.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:27:44.643 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:27:44.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:44.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:27:44.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:45.564 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:45.556+0000 7efcb15b78c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:27:45.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:45.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:45.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:27:45.791 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:27:45.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:45.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:27:45.935 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2827931620,v1:127.0.0.1:6811/2827931620] [v2:127.0.0.1:6812/2827931620,v1:127.0.0.1:6813/2827931620] exists,up 5b079d49-2751-48ad-83e1-9c407704f078 2026-02-17T11:27:45.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:27:45.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:27:45.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:27:45.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T11:27:45.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:57: TEST_recovery_scrub_1: run_osd td/osd-recovery-scrub 2 --osd_scrub_during_recovery=false 2026-02-17T11:27:45.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2026-02-17T11:27:45.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:27:45.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T11:27:45.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:27:45.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/2 2026-02-17T11:27:45.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=14e8264a-2160-44dd-98ca-5157ccfa6ac4 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2026-02-17T11:27:45.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:27:45.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:27:45.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:27:45.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/2' 2026-02-17T11:27:45.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/2/journal' 2026-02-17T11:27:45.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:27:45.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:27:45.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2026-02-17T11:27:45.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:27:45.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:27:45.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:27:45.941 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:27:45.941 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:27:45.941 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31716 2026-02-17T11:27:45.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.31716/$cluster-$name.asok' 2026-02-17T11:27:45.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31716/$cluster-$name.asok' 2026-02-17T11:27:45.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:27:45.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:27:45.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:27:45.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2026-02-17T11:27:45.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2026-02-17T11:27:45.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:27:45.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:27:45.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:27:45.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:27:45.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:27:45.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_scrub_during_recovery=false 2026-02-17T11:27:45.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/2 2026-02-17T11:27:45.942 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:27:45.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=bd313ccd-fa25-4b49-ab7e-1324de308505 2026-02-17T11:27:45.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 bd313ccd-fa25-4b49-ab7e-1324de308505' 2026-02-17T11:27:45.943 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 bd313ccd-fa25-4b49-ab7e-1324de308505 2026-02-17T11:27:45.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:27:45.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCxUJRpOKaAOBAAcO9SQFMBjFFHI5lOxsLFHA== 2026-02-17T11:27:45.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCxUJRpOKaAOBAAcO9SQFMBjFFHI5lOxsLFHA=="}' 2026-02-17T11:27:45.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new bd313ccd-fa25-4b49-ab7e-1324de308505 -i td/osd-recovery-scrub/2/new.json 2026-02-17T11:27:46.107 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:27:46.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/2/new.json 2026-02-17T11:27:46.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=14e8264a-2160-44dd-98ca-5157ccfa6ac4 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/2 --osd-journal=td/osd-recovery-scrub/2/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.31716/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --mkfs --key AQCxUJRpOKaAOBAAcO9SQFMBjFFHI5lOxsLFHA== --osd-uuid bd313ccd-fa25-4b49-ab7e-1324de308505 2026-02-17T11:27:46.133 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:46.124+0000 7f27fd3378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:46.134 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:46.128+0000 7f27fd3378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:46.135 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:46.128+0000 7f27fd3378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:46.135 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:46.128+0000 7f27fd3378c0 -1 bdev(0x564cc1c95c00 td/osd-recovery-scrub/2/block) open stat got: (1) Operation not permitted 2026-02-17T11:27:46.136 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:46.128+0000 7f27fd3378c0 -1 bluestore(td/osd-recovery-scrub/2) _read_fsid unparsable uuid 2026-02-17T11:27:48.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/2/keyring 2026-02-17T11:27:48.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:27:48.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T11:27:48.403 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T11:27:48.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:27:48.594 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T11:27:48.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T11:27:48.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=14e8264a-2160-44dd-98ca-5157ccfa6ac4 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/2 --osd-journal=td/osd-recovery-scrub/2/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.31716/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false 2026-02-17T11:27:48.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:27:48.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:27:48.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:27:48.617 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:48.608+0000 7f37cd8638c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:48.618 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:48.612+0000 7f37cd8638c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:48.620 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:48.612+0000 7f37cd8638c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:48.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T11:27:48.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:27:48.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:27:48.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:27:48.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:27:48.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:48.750 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:27:48.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:27:48.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:48.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:27:48.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:49.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:49.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:49.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:27:49.904 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:27:49.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:49.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:27:50.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:50.135 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:50.128+0000 7f37cd8638c0 -1 Falling back to public interface 2026-02-17T11:27:51.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:51.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:51.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:27:51.053 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:27:51.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:51.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:27:51.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:52.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:52.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:52.201 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:27:52.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:27:52.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:52.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:27:52.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:53.118 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:53.112+0000 7f37cd8638c0 -1 osd.2 0 log_to_monitors true 2026-02-17T11:27:53.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:53.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:53.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:27:53.349 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:27:53.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:53.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:27:53.496 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3009067208,v1:127.0.0.1:6819/3009067208] [v2:127.0.0.1:6820/3009067208,v1:127.0.0.1:6821/3009067208] exists,up bd313ccd-fa25-4b49-ab7e-1324de308505 2026-02-17T11:27:53.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:27:53.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:27:53.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:27:53.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:55: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T11:27:53.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:57: TEST_recovery_scrub_1: run_osd td/osd-recovery-scrub 3 --osd_scrub_during_recovery=false 2026-02-17T11:27:53.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-recovery-scrub 2026-02-17T11:27:53.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:27:53.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T11:27:53.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:27:53.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-recovery-scrub/3 2026-02-17T11:27:53.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=14e8264a-2160-44dd-98ca-5157ccfa6ac4 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq ' 2026-02-17T11:27:53.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:27:53.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:27:53.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:27:53.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-recovery-scrub/3' 2026-02-17T11:27:53.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-recovery-scrub/3/journal' 2026-02-17T11:27:53.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:27:53.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:27:53.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-recovery-scrub' 2026-02-17T11:27:53.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:27:53.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:27:53.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:27:53.506 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:27:53.506 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:27:53.506 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31716 2026-02-17T11:27:53.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.31716/$cluster-$name.asok' 2026-02-17T11:27:53.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31716/$cluster-$name.asok' 2026-02-17T11:27:53.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:27:53.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:27:53.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:27:53.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-recovery-scrub/$name.log' 2026-02-17T11:27:53.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-recovery-scrub/$name.pid' 2026-02-17T11:27:53.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:27:53.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:27:53.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:27:53.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:27:53.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:27:53.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_scrub_during_recovery=false 2026-02-17T11:27:53.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-recovery-scrub/3 2026-02-17T11:27:53.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:27:53.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=18c326fc-f28b-4aaf-84f1-2910c3549bc4 2026-02-17T11:27:53.509 INFO:tasks.workunit.client.0.trial128.stdout:add osd3 18c326fc-f28b-4aaf-84f1-2910c3549bc4 2026-02-17T11:27:53.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 18c326fc-f28b-4aaf-84f1-2910c3549bc4' 2026-02-17T11:27:53.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:27:53.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQC5UJRpCXldHhAAutS260Jp3tM3dA8TGYYmag== 2026-02-17T11:27:53.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQC5UJRpCXldHhAAutS260Jp3tM3dA8TGYYmag=="}' 2026-02-17T11:27:53.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 18c326fc-f28b-4aaf-84f1-2910c3549bc4 -i td/osd-recovery-scrub/3/new.json 2026-02-17T11:27:53.659 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:27:53.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-recovery-scrub/3/new.json 2026-02-17T11:27:53.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=14e8264a-2160-44dd-98ca-5157ccfa6ac4 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/3 --osd-journal=td/osd-recovery-scrub/3/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.31716/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --mkfs --key AQC5UJRpCXldHhAAutS260Jp3tM3dA8TGYYmag== --osd-uuid 18c326fc-f28b-4aaf-84f1-2910c3549bc4 2026-02-17T11:27:53.685 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:53.676+0000 7fde683598c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:53.687 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:53.680+0000 7fde683598c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:53.687 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:53.680+0000 7fde683598c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:53.688 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:53.680+0000 7fde683598c0 -1 bdev(0x556dc5967c00 td/osd-recovery-scrub/3/block) open stat got: (1) Operation not permitted 2026-02-17T11:27:53.688 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:53.680+0000 7fde683598c0 -1 bluestore(td/osd-recovery-scrub/3) _read_fsid unparsable uuid 2026-02-17T11:27:55.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-recovery-scrub/3/keyring 2026-02-17T11:27:55.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:27:55.960 INFO:tasks.workunit.client.0.trial128.stdout:adding osd3 key to auth repository 2026-02-17T11:27:55.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T11:27:55.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-recovery-scrub/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:27:56.152 INFO:tasks.workunit.client.0.trial128.stdout:start osd.3 2026-02-17T11:27:56.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T11:27:56.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=14e8264a-2160-44dd-98ca-5157ccfa6ac4 --auth-supported=none --mon-host=127.0.0.1:7124 --osd-op-queue=wpq --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-recovery-scrub/3 --osd-journal=td/osd-recovery-scrub/3/journal --chdir= --run-dir=td/osd-recovery-scrub '--admin-socket=/tmp/ceph-asok.31716/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-recovery-scrub/$name.log' '--pid-file=td/osd-recovery-scrub/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false 2026-02-17T11:27:56.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:27:56.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:27:56.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:27:56.175 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:56.168+0000 7f213b8bb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:56.177 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:56.168+0000 7f213b8bb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:56.178 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:56.172+0000 7f213b8bb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:27:56.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T11:27:56.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:27:56.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:27:56.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:27:56.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:27:56.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:56.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:27:56.307 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:27:56.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:56.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:27:56.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:57.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:57.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:57.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:27:57.459 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:27:57.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:57.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:27:57.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:57.651 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:27:57.643+0000 7f213b8bb8c0 -1 Falling back to public interface 2026-02-17T11:27:58.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:58.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:58.608 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:27:58.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:27:58.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:58.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:27:58.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:27:59.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:27:59.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:27:59.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:27:59.759 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:27:59.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:27:59.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:27:59.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:28:00.615 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:00.607+0000 7f213b8bb8c0 -1 osd.3 0 log_to_monitors true 2026-02-17T11:28:00.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:28:00.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:28:00.910 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:28:00.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:28:00.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:28:00.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:28:01.057 INFO:tasks.workunit.client.0.trial128.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/73498218,v1:127.0.0.1:6827/73498218] [v2:127.0.0.1:6828/73498218,v1:127.0.0.1:6829/73498218] exists,up 18c326fc-f28b-4aaf-84f1-2910c3549bc4 2026-02-17T11:28:01.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:28:01.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:28:01.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:28:01.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:61: TEST_recovery_scrub_1: create_pool test 1 1 2026-02-17T11:28:01.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T11:28:01.253 INFO:tasks.workunit.client.0.trial128.stderr:pool 'test' created 2026-02-17T11:28:01.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:28:02.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:62: TEST_recovery_scrub_1: wait_for_clean 2026-02-17T11:28:02.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:28:02.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:28:02.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:28:02.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:28:02.268 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:28:02.268 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:28:02.269 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:28:02.269 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:28:02.269 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:28:02.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:28:02.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:28:02.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:28:02.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:28:02.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:28:02.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:28:02.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:28:02.527 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:28:02.527 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:28:02.527 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T11:28:02.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:28:02.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:28:02.527 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:28:02.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836486 2026-02-17T11:28:02.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836486 2026-02-17T11:28:02.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486' 2026-02-17T11:28:02.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:28:02.599 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:28:02.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-17T11:28:02.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-17T11:28:02.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-38654705669' 2026-02-17T11:28:02.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:28:02.672 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:28:02.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542147 2026-02-17T11:28:02.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542147 2026-02-17T11:28:02.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-38654705669 2-60129542147' 2026-02-17T11:28:02.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:28:02.742 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:28:02.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411330 2026-02-17T11:28:02.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411330 2026-02-17T11:28:02.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-38654705669 2-60129542147 3-77309411330' 2026-02-17T11:28:02.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:28:02.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836486 2026-02-17T11:28:02.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:28:02.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:28:02.819 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836486 2026-02-17T11:28:02.819 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:28:02.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836486 2026-02-17T11:28:02.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-17T11:28:02.821 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836486 2026-02-17T11:28:02.821 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:28:02.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-17T11:28:02.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:28:02.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-17T11:28:02.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:28:02.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:28:02.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-17T11:28:02.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:28:02.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-17T11:28:02.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-17T11:28:02.981 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705669 2026-02-17T11:28:02.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:28:03.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705668 -lt 38654705669 2026-02-17T11:28:03.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:28:04.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:28:04.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:28:04.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705668 -lt 38654705669 2026-02-17T11:28:04.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:28:05.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:28:05.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:28:05.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-17T11:28:05.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:28:05.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542147 2026-02-17T11:28:05.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:28:05.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:28:05.455 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542147 2026-02-17T11:28:05.455 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:28:05.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542147 2026-02-17T11:28:05.456 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542147 2026-02-17T11:28:05.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-17T11:28:05.457 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:28:05.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542148 -lt 60129542147 2026-02-17T11:28:05.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:28:05.622 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-77309411330 2026-02-17T11:28:05.622 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:28:05.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:28:05.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-77309411330 2026-02-17T11:28:05.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:28:05.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411330 2026-02-17T11:28:05.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 77309411330' 2026-02-17T11:28:05.625 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.3 seq 77309411330 2026-02-17T11:28:05.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:28:05.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411330 -lt 77309411330 2026-02-17T11:28:05.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:28:05.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:28:05.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:28:05.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:28:05.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:28:05.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:28:05.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:28:05.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:28:05.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:28:05.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:28:05.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:28:06.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:28:06.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:28:06.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:28:06.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:28:06.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:28:06.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:28:06.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:28:06.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:63: TEST_recovery_scrub_1: ceph osd dump 2026-02-17T11:28:06.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:63: TEST_recovery_scrub_1: grep '^pool.*['\'']test['\'']' 2026-02-17T11:28:06.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:63: TEST_recovery_scrub_1: awk '{ print $2 }' 2026-02-17T11:28:06.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:63: TEST_recovery_scrub_1: poolid=1 2026-02-17T11:28:06.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:65: TEST_recovery_scrub_1: ceph pg dump pgs 2026-02-17T11:28:06.596 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T11:28:06.596 INFO:tasks.workunit.client.0.trial128.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-17T11:28:01.826334+0000 0'0 21:9 [1] 1 [1] 1 0'0 2026-02-17T11:28:01.248445+0000 0'0 2026-02-17T11:28:01.248445+0000 0 0 periodic scrub scheduled @ 2026-02-18T20:22:29.074864+0000 0 0 2026-02-17T11:28:06.596 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:28:06.597 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T11:28:06.597 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:28:06.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:67: TEST_recovery_scrub_1: dd if=/dev/urandom of=testdata.31716 bs=1M count=50 2026-02-17T11:28:06.746 INFO:tasks.workunit.client.0.trial128.stderr:50+0 records in 2026-02-17T11:28:06.746 INFO:tasks.workunit.client.0.trial128.stderr:50+0 records out 2026-02-17T11:28:06.747 INFO:tasks.workunit.client.0.trial128.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.141121 s, 372 MB/s 2026-02-17T11:28:06.747 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: seq 1 100 2026-02-17T11:28:06.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:06.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj1 testdata.31716 2026-02-17T11:28:06.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:06.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj2 testdata.31716 2026-02-17T11:28:06.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:06.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj3 testdata.31716 2026-02-17T11:28:06.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:06.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj4 testdata.31716 2026-02-17T11:28:07.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:07.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj5 testdata.31716 2026-02-17T11:28:07.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:07.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj6 testdata.31716 2026-02-17T11:28:07.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:07.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj7 testdata.31716 2026-02-17T11:28:07.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:07.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj8 testdata.31716 2026-02-17T11:28:07.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:07.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj9 testdata.31716 2026-02-17T11:28:07.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:07.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj10 testdata.31716 2026-02-17T11:28:07.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:07.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj11 testdata.31716 2026-02-17T11:28:07.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:07.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj12 testdata.31716 2026-02-17T11:28:07.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:07.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj13 testdata.31716 2026-02-17T11:28:07.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:07.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj14 testdata.31716 2026-02-17T11:28:07.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:07.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj15 testdata.31716 2026-02-17T11:28:07.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:07.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj16 testdata.31716 2026-02-17T11:28:08.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:08.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj17 testdata.31716 2026-02-17T11:28:08.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:08.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj18 testdata.31716 2026-02-17T11:28:08.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:08.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj19 testdata.31716 2026-02-17T11:28:08.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:08.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj20 testdata.31716 2026-02-17T11:28:08.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:08.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj21 testdata.31716 2026-02-17T11:28:08.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:08.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj22 testdata.31716 2026-02-17T11:28:08.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:08.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj23 testdata.31716 2026-02-17T11:28:08.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:08.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj24 testdata.31716 2026-02-17T11:28:08.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:08.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj25 testdata.31716 2026-02-17T11:28:08.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:08.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj26 testdata.31716 2026-02-17T11:28:08.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:08.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj27 testdata.31716 2026-02-17T11:28:08.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:08.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj28 testdata.31716 2026-02-17T11:28:09.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:09.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj29 testdata.31716 2026-02-17T11:28:09.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:09.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj30 testdata.31716 2026-02-17T11:28:09.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:09.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj31 testdata.31716 2026-02-17T11:28:09.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:09.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj32 testdata.31716 2026-02-17T11:28:09.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:09.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj33 testdata.31716 2026-02-17T11:28:09.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:09.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj34 testdata.31716 2026-02-17T11:28:09.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:09.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj35 testdata.31716 2026-02-17T11:28:09.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:09.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj36 testdata.31716 2026-02-17T11:28:09.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:09.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj37 testdata.31716 2026-02-17T11:28:09.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:09.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj38 testdata.31716 2026-02-17T11:28:09.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:09.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj39 testdata.31716 2026-02-17T11:28:09.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:09.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj40 testdata.31716 2026-02-17T11:28:10.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:10.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj41 testdata.31716 2026-02-17T11:28:10.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:10.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj42 testdata.31716 2026-02-17T11:28:10.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:10.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj43 testdata.31716 2026-02-17T11:28:10.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:10.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj44 testdata.31716 2026-02-17T11:28:10.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:10.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj45 testdata.31716 2026-02-17T11:28:10.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:10.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj46 testdata.31716 2026-02-17T11:28:10.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:10.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj47 testdata.31716 2026-02-17T11:28:10.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:10.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj48 testdata.31716 2026-02-17T11:28:10.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:10.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj49 testdata.31716 2026-02-17T11:28:10.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:10.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj50 testdata.31716 2026-02-17T11:28:10.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:10.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj51 testdata.31716 2026-02-17T11:28:10.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:10.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj52 testdata.31716 2026-02-17T11:28:11.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:11.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj53 testdata.31716 2026-02-17T11:28:11.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:11.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj54 testdata.31716 2026-02-17T11:28:11.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:11.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj55 testdata.31716 2026-02-17T11:28:11.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:11.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj56 testdata.31716 2026-02-17T11:28:11.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:11.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj57 testdata.31716 2026-02-17T11:28:11.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:11.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj58 testdata.31716 2026-02-17T11:28:11.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:11.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj59 testdata.31716 2026-02-17T11:28:11.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:11.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj60 testdata.31716 2026-02-17T11:28:11.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:11.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj61 testdata.31716 2026-02-17T11:28:11.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:11.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj62 testdata.31716 2026-02-17T11:28:12.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:12.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj63 testdata.31716 2026-02-17T11:28:12.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:12.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj64 testdata.31716 2026-02-17T11:28:12.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:12.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj65 testdata.31716 2026-02-17T11:28:12.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:12.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj66 testdata.31716 2026-02-17T11:28:12.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:12.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj67 testdata.31716 2026-02-17T11:28:12.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:12.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj68 testdata.31716 2026-02-17T11:28:12.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:12.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj69 testdata.31716 2026-02-17T11:28:12.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:12.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj70 testdata.31716 2026-02-17T11:28:12.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:12.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj71 testdata.31716 2026-02-17T11:28:12.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:12.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj72 testdata.31716 2026-02-17T11:28:13.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:13.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj73 testdata.31716 2026-02-17T11:28:13.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:13.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj74 testdata.31716 2026-02-17T11:28:13.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:13.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj75 testdata.31716 2026-02-17T11:28:13.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:13.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj76 testdata.31716 2026-02-17T11:28:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:13.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj77 testdata.31716 2026-02-17T11:28:13.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:13.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj78 testdata.31716 2026-02-17T11:28:13.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:13.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj79 testdata.31716 2026-02-17T11:28:13.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:13.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj80 testdata.31716 2026-02-17T11:28:13.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:13.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj81 testdata.31716 2026-02-17T11:28:13.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:13.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj82 testdata.31716 2026-02-17T11:28:13.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:13.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj83 testdata.31716 2026-02-17T11:28:13.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:13.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj84 testdata.31716 2026-02-17T11:28:14.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:14.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj85 testdata.31716 2026-02-17T11:28:14.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:14.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj86 testdata.31716 2026-02-17T11:28:14.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:14.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj87 testdata.31716 2026-02-17T11:28:14.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:14.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj88 testdata.31716 2026-02-17T11:28:14.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:14.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj89 testdata.31716 2026-02-17T11:28:14.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:14.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj90 testdata.31716 2026-02-17T11:28:14.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:14.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj91 testdata.31716 2026-02-17T11:28:14.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:14.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj92 testdata.31716 2026-02-17T11:28:14.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:14.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj93 testdata.31716 2026-02-17T11:28:14.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:14.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj94 testdata.31716 2026-02-17T11:28:14.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:14.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj95 testdata.31716 2026-02-17T11:28:14.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:14.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj96 testdata.31716 2026-02-17T11:28:15.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:15.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj97 testdata.31716 2026-02-17T11:28:15.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:15.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj98 testdata.31716 2026-02-17T11:28:15.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:15.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj99 testdata.31716 2026-02-17T11:28:15.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:68: TEST_recovery_scrub_1: for i in $(seq 1 $OBJECTS) 2026-02-17T11:28:15.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:70: TEST_recovery_scrub_1: rados -p test put obj100 testdata.31716 2026-02-17T11:28:15.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:72: TEST_recovery_scrub_1: rm -f testdata.31716 2026-02-17T11:28:15.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:74: TEST_recovery_scrub_1: ceph osd pool set test size 4 2026-02-17T11:28:15.656 INFO:tasks.workunit.client.0.trial128.stderr:set pool 1 size to 4 2026-02-17T11:28:15.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:77: TEST_recovery_scrub_1: set -o pipefail 2026-02-17T11:28:15.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:78: TEST_recovery_scrub_1: count=0 2026-02-17T11:28:15.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2026-02-17T11:28:15.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2026-02-17T11:28:15.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2026-02-17T11:28:15.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2026-02-17T11:28:15.808 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:28:15.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:86: TEST_recovery_scrub_1: sleep 2 2026-02-17T11:28:17.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:87: TEST_recovery_scrub_1: test 0 -eq 10 2026-02-17T11:28:17.819 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: expr 0 + 1 2026-02-17T11:28:17.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: count=1 2026-02-17T11:28:17.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2026-02-17T11:28:17.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2026-02-17T11:28:17.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2026-02-17T11:28:17.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2026-02-17T11:28:17.972 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:28:17.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:86: TEST_recovery_scrub_1: sleep 2 2026-02-17T11:28:19.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:87: TEST_recovery_scrub_1: test 1 -eq 10 2026-02-17T11:28:19.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: expr 1 + 1 2026-02-17T11:28:19.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: count=2 2026-02-17T11:28:19.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2026-02-17T11:28:19.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2026-02-17T11:28:19.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2026-02-17T11:28:19.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2026-02-17T11:28:20.137 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:28:20.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:86: TEST_recovery_scrub_1: sleep 2 2026-02-17T11:28:22.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:87: TEST_recovery_scrub_1: test 2 -eq 10 2026-02-17T11:28:22.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: expr 2 + 1 2026-02-17T11:28:22.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: count=3 2026-02-17T11:28:22.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2026-02-17T11:28:22.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2026-02-17T11:28:22.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2026-02-17T11:28:22.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2026-02-17T11:28:22.308 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:28:22.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:86: TEST_recovery_scrub_1: sleep 2 2026-02-17T11:28:24.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:87: TEST_recovery_scrub_1: test 3 -eq 10 2026-02-17T11:28:24.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: expr 3 + 1 2026-02-17T11:28:24.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:92: TEST_recovery_scrub_1: count=4 2026-02-17T11:28:24.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:79: TEST_recovery_scrub_1: true 2026-02-17T11:28:24.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:81: TEST_recovery_scrub_1: ceph --format json pg dump pgs 2026-02-17T11:28:24.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: jq '.pg_stats | [.[] | .state | contains("recovering")]' 2026-02-17T11:28:24.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:82: TEST_recovery_scrub_1: grep -q true 2026-02-17T11:28:24.472 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:28:24.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:84: TEST_recovery_scrub_1: break 2026-02-17T11:28:24.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:94: TEST_recovery_scrub_1: set +o pipefail 2026-02-17T11:28:24.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:95: TEST_recovery_scrub_1: ceph pg dump pgs 2026-02-17T11:28:24.630 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T11:28:24.630 INFO:tasks.workunit.client.0.trial128.stdout:1.0 100 0 249 0 0 5242880000 0 0 1300 0 1300 active+recovering+undersized+degraded+remapped 2026-02-17T11:28:18.031356+0000 22'1300 25:1391 [1,0,2,3] 1 [1,0] 1 0'0 2026-02-17T11:28:01.248445+0000 0'0 2026-02-17T11:28:01.248445+0000 0 0 no scrub is scheduled 0 0 2026-02-17T11:28:24.630 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:28:24.630 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T11:28:24.630 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:28:24.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:97: TEST_recovery_scrub_1: sleep 10 2026-02-17T11:28:34.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:99: TEST_recovery_scrub_1: kill_daemons td/osd-recovery-scrub 2026-02-17T11:28:34.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:28:34.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:28:34.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:28:34.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:28:34.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:28:42.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:28:42.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:101: TEST_recovery_scrub_1: declare -a err_strings 2026-02-17T11:28:42.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:102: TEST_recovery_scrub_1: err_strings[0]='recovery in progress. Only high priority scrubs allowed.' 2026-02-17T11:28:42.958 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: expr 4 - 1 2026-02-17T11:28:42.959 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: seq 0 3 2026-02-17T11:28:42.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T11:28:42.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:106: TEST_recovery_scrub_1: grep 'recovery in progress. Only high priority scrubs allowed.' td/osd-recovery-scrub/osd.0.log 2026-02-17T11:28:42.965 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:22.059+0000 7fca1e3cd640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.965 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:23.095+0000 7fca1e3cd640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.965 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:24.131+0000 7fca1e3cd640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.965 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:31.271+0000 7fca1e3cd640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.965 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:32.223+0000 7fca1e3cd640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.965 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:35.284+0000 7fca1e3cd640 15 osd.0 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T11:28:42.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:106: TEST_recovery_scrub_1: grep 'recovery in progress. Only high priority scrubs allowed.' td/osd-recovery-scrub/osd.1.log 2026-02-17T11:28:42.975 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:21.943+0000 7efcaa56b640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.975 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:22.991+0000 7efcaa56b640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.975 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:23.979+0000 7efcaa56b640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.975 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:24.939+0000 7efcaa56b640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.975 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:25.943+0000 7efcaa56b640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.975 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:34.904+0000 7efcaa56b640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.975 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:36.828+0000 7efcaa56b640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.975 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:38.844+0000 7efcaa56b640 15 osd.1 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T11:28:42.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:106: TEST_recovery_scrub_1: grep 'recovery in progress. Only high priority scrubs allowed.' td/osd-recovery-scrub/osd.2.log 2026-02-17T11:28:42.978 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:23.235+0000 7f37c6817640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.978 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:24.259+0000 7f37c6817640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.978 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:25.259+0000 7f37c6817640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.978 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:26.243+0000 7f37c6817640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.978 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:30.275+0000 7f37c6817640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.978 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:34.223+0000 7f37c6817640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.979 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:36.192+0000 7f37c6817640 15 osd.2 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:104: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T11:28:42.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:106: TEST_recovery_scrub_1: grep 'recovery in progress. Only high priority scrubs allowed.' td/osd-recovery-scrub/osd.3.log 2026-02-17T11:28:42.983 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:18.755+0000 7f213486f640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.983 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:21.711+0000 7f213486f640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.983 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:24.787+0000 7f213486f640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.983 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:27.711+0000 7f213486f640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.983 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:28.711+0000 7f213486f640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.983 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:33.623+0000 7f213486f640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.983 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:35.604+0000 7f213486f640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.983 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:28:37.556+0000 7f213486f640 15 osd.3 osd-scrub:restrictions_on_scrubbing: recovery in progress. Only high priority scrubs allowed. 2026-02-17T11:28:42.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:108: TEST_recovery_scrub_1: for err_string in "${err_strings[@]}" 2026-02-17T11:28:42.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:110: TEST_recovery_scrub_1: found=false 2026-02-17T11:28:42.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:111: TEST_recovery_scrub_1: count=0 2026-02-17T11:28:42.984 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: expr 4 - 1 2026-02-17T11:28:42.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: seq 0 3 2026-02-17T11:28:42.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T11:28:42.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:114: TEST_recovery_scrub_1: grep -q 'recovery in progress. Only high priority scrubs allowed.' td/osd-recovery-scrub/osd.0.log 2026-02-17T11:28:42.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:116: TEST_recovery_scrub_1: found=true 2026-02-17T11:28:42.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: expr 0 + 1 2026-02-17T11:28:42.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: count=1 2026-02-17T11:28:42.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T11:28:42.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:114: TEST_recovery_scrub_1: grep -q 'recovery in progress. Only high priority scrubs allowed.' td/osd-recovery-scrub/osd.1.log 2026-02-17T11:28:42.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:116: TEST_recovery_scrub_1: found=true 2026-02-17T11:28:42.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: expr 1 + 1 2026-02-17T11:28:42.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: count=2 2026-02-17T11:28:43.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T11:28:43.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:114: TEST_recovery_scrub_1: grep -q 'recovery in progress. Only high priority scrubs allowed.' td/osd-recovery-scrub/osd.2.log 2026-02-17T11:28:43.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:116: TEST_recovery_scrub_1: found=true 2026-02-17T11:28:43.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: expr 2 + 1 2026-02-17T11:28:43.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: count=3 2026-02-17T11:28:43.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:112: TEST_recovery_scrub_1: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T11:28:43.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:114: TEST_recovery_scrub_1: grep -q 'recovery in progress. Only high priority scrubs allowed.' td/osd-recovery-scrub/osd.3.log 2026-02-17T11:28:43.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:116: TEST_recovery_scrub_1: found=true 2026-02-17T11:28:43.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: expr 3 + 1 2026-02-17T11:28:43.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:117: TEST_recovery_scrub_1: count=4 2026-02-17T11:28:43.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:120: TEST_recovery_scrub_1: '[' true = false ']' 2026-02-17T11:28:43.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:124: TEST_recovery_scrub_1: '[' 4 -eq 4 ']' 2026-02-17T11:28:43.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:127: TEST_recovery_scrub_1: teardown td/osd-recovery-scrub 2026-02-17T11:28:43.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-recovery-scrub 2026-02-17T11:28:43.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:28:43.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-recovery-scrub KILL 2026-02-17T11:28:43.007 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:28:43.008 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:28:43.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:28:43.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:28:43.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:28:43.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:28:43.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:28:43.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:28:43.014 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:28:43.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:28:43.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:28:43.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:28:43.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:28:43.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:28:43.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:28:43.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:28:43.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:28:43.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:28:43.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-recovery-scrub 2026-02-17T11:28:43.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:28:43.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:28:43.065 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31716 2026-02-17T11:28:43.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.31716 2026-02-17T11:28:43.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:28:43.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:28:43.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:129: TEST_recovery_scrub_1: '[' 0 '!=' 0 ']' 2026-02-17T11:28:43.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:135: TEST_recovery_scrub_1: echo 'TEST PASSED' 2026-02-17T11:28:43.066 INFO:tasks.workunit.client.0.trial128.stdout:TEST PASSED 2026-02-17T11:28:43.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh:136: TEST_recovery_scrub_1: return 0 2026-02-17T11:28:43.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T11:28:43.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-recovery-scrub 0 2026-02-17T11:28:43.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-recovery-scrub 2026-02-17T11:28:43.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T11:28:43.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-recovery-scrub KILL 2026-02-17T11:28:43.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:28:43.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:28:43.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:28:43.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:28:43.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:28:43.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:28:43.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:28:43.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:28:43.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:28:43.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:28:43.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:28:43.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:28:43.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:28:43.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:28:43.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:28:43.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:28:43.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:28:43.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T11:28:43.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-recovery-scrub 2026-02-17T11:28:43.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:28:43.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:28:43.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.31716 2026-02-17T11:28:43.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.31716 2026-02-17T11:28:43.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:28:43.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:28:43.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T11:28:43.079 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T11:28:43.079 DEBUG:teuthology.orchestra.run.trial128:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T11:28:43.132 INFO:tasks.workunit:Running workunit scrub/osd-scrub-dump.sh... 2026-02-17T11:28:43.133 DEBUG:teuthology.orchestra.run.trial128:workunit test scrub/osd-scrub-dump.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3c96dfb3f50b3d02598a82af7104764bfdcd3919 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh 2026-02-17T11:28:43.186 INFO:tasks.workunit.client.0.trial128.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T11:28:43.190 INFO:tasks.workunit.client.0.trial128.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T11:28:43.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T11:28:43.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T11:28:43.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T11:28:43.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T11:28:43.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T11:28:43.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T11:28:43.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T11:28:43.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T11:28:43.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-scrub-dump 2026-02-17T11:28:43.192 INFO:tasks.workunit.client.0.trial128.stdout:This test is disabled 2026-02-17T11:28:43.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:30: run: echo 'This test is disabled' 2026-02-17T11:28:43.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-dump.sh:31: run: return 0 2026-02-17T11:28:43.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T11:28:43.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-scrub-dump 0 2026-02-17T11:28:43.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-dump 2026-02-17T11:28:43.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T11:28:43.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-dump KILL 2026-02-17T11:28:43.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:28:43.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:28:43.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:28:43.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:28:43.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:28:43.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:28:43.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:28:43.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:28:43.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:28:43.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:28:43.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:28:43.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:28:43.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:28:43.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:28:43.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:28:43.198 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:28:43.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:28:43.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T11:28:43.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-dump 2026-02-17T11:28:43.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:28:43.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:28:43.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40862 2026-02-17T11:28:43.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40862 2026-02-17T11:28:43.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:28:43.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:28:43.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T11:28:43.202 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T11:28:43.202 DEBUG:teuthology.orchestra.run.trial128:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T11:28:43.253 INFO:tasks.workunit:Running workunit scrub/osd-scrub-repair.sh... 2026-02-17T11:28:43.253 DEBUG:teuthology.orchestra.run.trial128:workunit test scrub/osd-scrub-repair.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3c96dfb3f50b3d02598a82af7104764bfdcd3919 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh 2026-02-17T11:28:43.303 INFO:tasks.workunit.client.0.trial128.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh 2026-02-17T11:28:43.303 INFO:tasks.workunit.client.0.trial128.stderr:++ TIMEOUT=300 2026-02-17T11:28:43.303 INFO:tasks.workunit.client.0.trial128.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2026-02-17T11:28:43.304 INFO:tasks.workunit.client.0.trial128.stderr:++ MAX_TIMEOUT=15 2026-02-17T11:28:43.304 INFO:tasks.workunit.client.0.trial128.stderr:++ PG_NUM=4 2026-02-17T11:28:43.304 INFO:tasks.workunit.client.0.trial128.stderr:++ TMPDIR=/tmp 2026-02-17T11:28:43.304 INFO:tasks.workunit.client.0.trial128.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2026-02-17T11:28:43.304 INFO:tasks.workunit.client.0.trial128.stderr:++ TESTDIR=/home/ubuntu/cephtest 2026-02-17T11:28:43.304 INFO:tasks.workunit.client.0.trial128.stderr:++ type xmlstarlet 2026-02-17T11:28:43.304 INFO:tasks.workunit.client.0.trial128.stderr:++ XMLSTARLET=xmlstarlet 2026-02-17T11:28:43.304 INFO:tasks.workunit.client.0.trial128.stderr:+++ uname 2026-02-17T11:28:43.305 INFO:tasks.workunit.client.0.trial128.stderr:++ '[' Linux = FreeBSD ']' 2026-02-17T11:28:43.305 INFO:tasks.workunit.client.0.trial128.stderr:++ SED=sed 2026-02-17T11:28:43.305 INFO:tasks.workunit.client.0.trial128.stderr:++ AWK=awk 2026-02-17T11:28:43.305 INFO:tasks.workunit.client.0.trial128.stderr:+++ stty -a 2026-02-17T11:28:43.305 INFO:tasks.workunit.client.0.trial128.stderr:+++ head -1 2026-02-17T11:28:43.305 INFO:tasks.workunit.client.0.trial128.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2026-02-17T11:28:43.306 INFO:tasks.workunit.client.0.trial128.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T11:28:43.306 INFO:tasks.workunit.client.0.trial128.stderr:++ termwidth= 2026-02-17T11:28:43.306 INFO:tasks.workunit.client.0.trial128.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2026-02-17T11:28:43.306 INFO:tasks.workunit.client.0.trial128.stderr:++ DIFFCOLOPTS='-y ' 2026-02-17T11:28:43.306 INFO:tasks.workunit.client.0.trial128.stderr:++ KERNCORE=kernel.core_pattern 2026-02-17T11:28:43.307 INFO:tasks.workunit.client.0.trial128.stderr:++ EXTRA_OPTS= 2026-02-17T11:28:43.308 INFO:tasks.workunit.client.0.trial128.stderr:++ test '' = TESTS 2026-02-17T11:28:43.308 INFO:tasks.workunit.client.0.trial128.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh 2026-02-17T11:28:43.308 INFO:tasks.workunit.client.0.trial128.stderr:++ uname 2026-02-17T11:28:43.309 INFO:tasks.workunit.client.0.trial128.stderr:+ '[' Linux = FreeBSD ']' 2026-02-17T11:28:43.309 INFO:tasks.workunit.client.0.trial128.stderr:+ use_ec_overwrite=true 2026-02-17T11:28:43.309 INFO:tasks.workunit.client.0.trial128.stderr:+ getjson=no 2026-02-17T11:28:43.310 INFO:tasks.workunit.client.0.trial128.stderr:+ jqfilter='def walk(f): 2026-02-17T11:28:43.310 INFO:tasks.workunit.client.0.trial128.stderr: . as $in 2026-02-17T11:28:43.310 INFO:tasks.workunit.client.0.trial128.stderr: | if type == "object" then 2026-02-17T11:28:43.310 INFO:tasks.workunit.client.0.trial128.stderr: reduce keys[] as $key 2026-02-17T11:28:43.310 INFO:tasks.workunit.client.0.trial128.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T11:28:43.310 INFO:tasks.workunit.client.0.trial128.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T11:28:43.310 INFO:tasks.workunit.client.0.trial128.stderr: else f 2026-02-17T11:28:43.310 INFO:tasks.workunit.client.0.trial128.stderr: end; 2026-02-17T11:28:43.310 INFO:tasks.workunit.client.0.trial128.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T11:28:43.310 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T11:28:43.310 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T11:28:43.310 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T11:28:43.310 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-17T11:28:43.311 INFO:tasks.workunit.client.0.trial128.stderr:+ sortkeys='import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T11:28:43.314 INFO:tasks.workunit.client.0.trial128.stderr:+ main osd-scrub-repair 2026-02-17T11:28:43.314 INFO:tasks.workunit.client.0.trial128.stderr:+ local dir=td/osd-scrub-repair 2026-02-17T11:28:43.315 INFO:tasks.workunit.client.0.trial128.stderr:+ shift 2026-02-17T11:28:43.315 INFO:tasks.workunit.client.0.trial128.stderr:+ shopt -s -o xtrace 2026-02-17T11:28:43.315 INFO:tasks.workunit.client.0.trial128.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T11:28:43.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T11:28:43.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T11:28:43.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T11:28:43.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T11:28:43.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T11:28:43.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T11:28:43.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T11:28:43.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T11:28:43.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-scrub-repair 2026-02-17T11:28:43.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:52: run: local dir=td/osd-scrub-repair 2026-02-17T11:28:43.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:53: run: shift 2026-02-17T11:28:43.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:55: run: export CEPH_MON=127.0.0.1:7107 2026-02-17T11:28:43.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:55: run: CEPH_MON=127.0.0.1:7107 2026-02-17T11:28:43.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:56: run: export CEPH_ARGS 2026-02-17T11:28:43.316 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:57: run: uuidgen 2026-02-17T11:28:43.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:57: run: CEPH_ARGS+='--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none ' 2026-02-17T11:28:43.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:58: run: CEPH_ARGS+='--mon-host=127.0.0.1:7107 ' 2026-02-17T11:28:43.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:59: run: CEPH_ARGS+='--osd-skip-data-digest=false ' 2026-02-17T11:28:43.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:61: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-17T11:28:43.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:62: run: set 2026-02-17T11:28:43.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:62: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T11:28:43.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:62: run: local 'funcs=TEST_allow_repair_during_recovery 2026-02-17T11:28:43.320 INFO:tasks.workunit.client.0.trial128.stderr:TEST_auto_repair_bluestore_basic 2026-02-17T11:28:43.320 INFO:tasks.workunit.client.0.trial128.stderr:TEST_auto_repair_bluestore_failed 2026-02-17T11:28:43.320 INFO:tasks.workunit.client.0.trial128.stderr:TEST_auto_repair_bluestore_failed_norecov 2026-02-17T11:28:43.321 INFO:tasks.workunit.client.0.trial128.stderr:TEST_auto_repair_bluestore_scrub 2026-02-17T11:28:43.321 INFO:tasks.workunit.client.0.trial128.stderr:TEST_auto_repair_bluestore_tag 2026-02-17T11:28:43.321 INFO:tasks.workunit.client.0.trial128.stderr:TEST_auto_repair_erasure_coded_appends 2026-02-17T11:28:43.321 INFO:tasks.workunit.client.0.trial128.stderr:TEST_auto_repair_erasure_coded_overwrites 2026-02-17T11:28:43.321 INFO:tasks.workunit.client.0.trial128.stderr:TEST_corrupt_and_repair_jerasure_appends 2026-02-17T11:28:43.321 INFO:tasks.workunit.client.0.trial128.stderr:TEST_corrupt_and_repair_jerasure_overwrites 2026-02-17T11:28:43.321 INFO:tasks.workunit.client.0.trial128.stderr:TEST_corrupt_and_repair_lrc_appends 2026-02-17T11:28:43.321 INFO:tasks.workunit.client.0.trial128.stderr:TEST_corrupt_and_repair_lrc_overwrites 2026-02-17T11:28:43.321 INFO:tasks.workunit.client.0.trial128.stderr:TEST_corrupt_and_repair_replicated 2026-02-17T11:28:43.321 INFO:tasks.workunit.client.0.trial128.stderr:TEST_corrupt_scrub_erasure_appends 2026-02-17T11:28:43.321 INFO:tasks.workunit.client.0.trial128.stderr:TEST_corrupt_scrub_erasure_overwrites 2026-02-17T11:28:43.321 INFO:tasks.workunit.client.0.trial128.stderr:TEST_corrupt_scrub_replicated 2026-02-17T11:28:43.321 INFO:tasks.workunit.client.0.trial128.stderr:TEST_corrupt_snapset_scrub_rep 2026-02-17T11:28:43.322 INFO:tasks.workunit.client.0.trial128.stderr:TEST_list_missing_erasure_coded_appends 2026-02-17T11:28:43.322 INFO:tasks.workunit.client.0.trial128.stderr:TEST_list_missing_erasure_coded_overwrites 2026-02-17T11:28:43.322 INFO:tasks.workunit.client.0.trial128.stderr:TEST_periodic_scrub_replicated 2026-02-17T11:28:43.322 INFO:tasks.workunit.client.0.trial128.stderr:TEST_repair_stats 2026-02-17T11:28:43.322 INFO:tasks.workunit.client.0.trial128.stderr:TEST_repair_stats_ec 2026-02-17T11:28:43.322 INFO:tasks.workunit.client.0.trial128.stderr:TEST_request_scrub_priority 2026-02-17T11:28:43.322 INFO:tasks.workunit.client.0.trial128.stderr:TEST_scrub_warning 2026-02-17T11:28:43.322 INFO:tasks.workunit.client.0.trial128.stderr:TEST_skip_non_repair_during_recovery 2026-02-17T11:28:43.322 INFO:tasks.workunit.client.0.trial128.stderr:TEST_unfound_erasure_coded_appends 2026-02-17T11:28:43.322 INFO:tasks.workunit.client.0.trial128.stderr:TEST_unfound_erasure_coded_overwrites' 2026-02-17T11:28:43.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T11:28:43.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T11:28:43.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T11:28:43.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T11:28:43.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:28:43.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:28:43.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:28:43.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:28:43.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:28:43.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:28:43.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:28:43.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:28:43.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:28:43.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:28:43.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:28:43.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:28:43.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:28:43.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:28:43.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:28:43.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:28:43.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:28:43.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:28:43.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:28:43.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:28:43.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:28:43.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:28:43.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:28:43.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:28:43.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:28:43.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:28:43.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:28:43.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:28:43.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T11:28:43.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:28:43.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:28:43.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:28:43.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T11:28:43.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:28:43.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T11:28:43.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T11:28:43.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:28:43.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T11:28:43.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_allow_repair_during_recovery td/osd-scrub-repair 2026-02-17T11:28:43.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:114: TEST_allow_repair_during_recovery: local dir=td/osd-scrub-repair 2026-02-17T11:28:43.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:115: TEST_allow_repair_during_recovery: local poolname=rbd 2026-02-17T11:28:43.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:117: TEST_allow_repair_during_recovery: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-17T11:28:43.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T11:28:43.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T11:28:43.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T11:28:43.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T11:28:43.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T11:28:43.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-17T11:28:43.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T11:28:43.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:28:43.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:28:43.364 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:28:43.364 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:28:43.364 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:28:43.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:28:43.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T11:28:43.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T11:28:43.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T11:28:43.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:28:43.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:28:43.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T11:28:43.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T11:28:43.395 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:28:43.395 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:28:43.395 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:28:43.396 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:28:43.396 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:28:43.396 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:28:43.396 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:28:43.396 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:28:43.396 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T11:28:43.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T11:28:43.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:28:43.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:28:43.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T11:28:43.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T11:28:43.453 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:28:43.453 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:28:43.453 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:28:43.453 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:28:43.453 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:28:43.454 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:28:43.454 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:28:43.454 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:28:43.454 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T11:28:43.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:118: TEST_allow_repair_during_recovery: run_mgr td/osd-scrub-repair x 2026-02-17T11:28:43.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T11:28:43.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T11:28:43.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T11:28:43.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T11:28:43.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T11:28:43.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T11:28:43.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T11:28:43.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:28:43.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:28:43.601 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:28:43.601 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:28:43.601 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:28:43.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:28:43.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:28:43.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:28:43.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:119: TEST_allow_repair_during_recovery: run_osd td/osd-scrub-repair 0 --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true 2026-02-17T11:28:43.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:28:43.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:28:43.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T11:28:43.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:28:43.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:28:43.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:28:43.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:28:43.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:28:43.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:28:43.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:28:43.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:28:43.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:28:43.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:28:43.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:28:43.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:28:43.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:28:43.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:28:43.624 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:28:43.624 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:28:43.624 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:28:43.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:28:43.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:28:43.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:28:43.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:28:43.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:28:43.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:28:43.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:28:43.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:28:43.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:28:43.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:28:43.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:28:43.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:28:43.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true' 2026-02-17T11:28:43.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:28:43.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:28:43.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=1ab256e5-e915-4e27-9fd9-e42ee275fb60 2026-02-17T11:28:43.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 1ab256e5-e915-4e27-9fd9-e42ee275fb60' 2026-02-17T11:28:43.627 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 1ab256e5-e915-4e27-9fd9-e42ee275fb60 2026-02-17T11:28:43.627 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:28:43.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDrUJRpMyWoJRAA9D7dozfFy6Y0844YPpP3Bw== 2026-02-17T11:28:43.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDrUJRpMyWoJRAA9D7dozfFy6Y0844YPpP3Bw=="}' 2026-02-17T11:28:43.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 1ab256e5-e915-4e27-9fd9-e42ee275fb60 -i td/osd-scrub-repair/0/new.json 2026-02-17T11:28:43.726 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:28:43.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T11:28:43.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true --mkfs --key AQDrUJRpMyWoJRAA9D7dozfFy6Y0844YPpP3Bw== --osd-uuid 1ab256e5-e915-4e27-9fd9-e42ee275fb60 2026-02-17T11:28:43.751 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:43.744+0000 7f3b948168c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:28:43.753 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:43.748+0000 7f3b948168c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:28:43.754 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:43.748+0000 7f3b948168c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:28:43.754 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:43.748+0000 7f3b948168c0 -1 bdev(0x55a60269bc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:28:43.754 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:43.748+0000 7f3b948168c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T11:28:46.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T11:28:46.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:28:46.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T11:28:46.629 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:28:46.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:28:46.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T11:28:46.821 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:28:46.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true 2026-02-17T11:28:46.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:28:46.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:28:46.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:28:46.844 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:46.836+0000 7fa6a0ddc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:28:46.845 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:46.840+0000 7fa6a0ddc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:28:46.846 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:46.840+0000 7fa6a0ddc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:28:46.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T11:28:46.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:28:46.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:28:46.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:28:46.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:28:46.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:28:46.976 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:28:46.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:28:46.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:28:46.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:28:47.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:28:48.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:28:48.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:28:48.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:28:48.125 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:28:48.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:28:48.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:28:48.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:28:48.353 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:48.348+0000 7fa6a0ddc8c0 -1 Falling back to public interface 2026-02-17T11:28:49.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:28:49.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:28:49.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:28:49.274 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:28:49.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:28:49.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:28:49.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:28:50.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:28:50.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:28:50.422 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:28:50.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:28:50.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:28:50.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:28:50.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:28:51.357 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:51.353+0000 7fa6a0ddc8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:28:51.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:28:51.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:28:51.570 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:28:51.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:28:51.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:28:51.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:28:51.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:28:52.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:28:52.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:28:52.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:28:52.722 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:28:52.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:28:52.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:28:52.870 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2876045110,v1:127.0.0.1:6803/2876045110] [v2:127.0.0.1:6804/2876045110,v1:127.0.0.1:6805/2876045110] exists,up 1ab256e5-e915-4e27-9fd9-e42ee275fb60 2026-02-17T11:28:52.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:28:52.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:28:52.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:28:52.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:122: TEST_allow_repair_during_recovery: run_osd td/osd-scrub-repair 1 --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true 2026-02-17T11:28:52.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:28:52.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:28:52.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T11:28:52.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:28:52.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:28:52.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:28:52.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:28:52.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:28:52.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:28:52.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:28:52.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:28:52.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:28:52.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:28:52.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:28:52.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:28:52.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:28:52.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:28:52.872 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:28:52.872 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:28:52.872 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:28:52.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:28:52.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:28:52.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:28:52.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:28:52.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:28:52.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:28:52.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:28:52.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:28:52.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:28:52.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:28:52.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:28:52.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:28:52.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true' 2026-02-17T11:28:52.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:28:52.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:28:52.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=8f5fb953-bba5-4b16-85f2-97c2129f3657 2026-02-17T11:28:52.875 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 8f5fb953-bba5-4b16-85f2-97c2129f3657 2026-02-17T11:28:52.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 8f5fb953-bba5-4b16-85f2-97c2129f3657' 2026-02-17T11:28:52.875 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:28:52.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQD0UJRpT2a/NBAAZXGzf3nBWhO1ZEfB1jSzRQ== 2026-02-17T11:28:52.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQD0UJRpT2a/NBAAZXGzf3nBWhO1ZEfB1jSzRQ=="}' 2026-02-17T11:28:52.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 8f5fb953-bba5-4b16-85f2-97c2129f3657 -i td/osd-scrub-repair/1/new.json 2026-02-17T11:28:53.036 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:28:53.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T11:28:53.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true --mkfs --key AQD0UJRpT2a/NBAAZXGzf3nBWhO1ZEfB1jSzRQ== --osd-uuid 8f5fb953-bba5-4b16-85f2-97c2129f3657 2026-02-17T11:28:53.062 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:53.057+0000 7f925a4c28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:28:53.063 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:53.061+0000 7f925a4c28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:28:53.065 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:53.061+0000 7f925a4c28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:28:53.065 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:53.061+0000 7f925a4c28c0 -1 bdev(0x55ef7fd11c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:28:53.065 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:53.061+0000 7f925a4c28c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T11:28:55.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T11:28:55.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:28:55.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T11:28:55.325 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:28:55.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:28:55.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T11:28:55.518 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:28:55.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true 2026-02-17T11:28:55.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:28:55.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:28:55.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:28:55.542 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:55.537+0000 7f08263c38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:28:55.543 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:55.541+0000 7f08263c38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:28:55.544 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:55.541+0000 7f08263c38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:28:55.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T11:28:55.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:28:55.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:28:55.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:28:55.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:28:55.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:28:55.672 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:28:55.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:28:55.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:28:55.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:28:55.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:28:56.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:28:56.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:28:56.819 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:28:56.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:28:56.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:28:56.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:28:56.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:28:57.033 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:28:57.029+0000 7f08263c38c0 -1 Falling back to public interface 2026-02-17T11:28:57.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:28:57.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:28:57.971 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:28:57.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:28:57.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:28:57.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:28:58.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:28:59.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:28:59.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:28:59.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:28:59.125 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:28:59.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:28:59.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:28:59.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:29:00.025 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:00.021+0000 7f08263c38c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:29:00.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:29:00.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:29:00.275 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:29:00.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:29:00.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:29:00.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:29:00.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:29:01.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:29:01.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:29:01.423 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:29:01.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:29:01.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:29:01.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:29:01.571 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3681890460,v1:127.0.0.1:6811/3681890460] [v2:127.0.0.1:6812/3681890460,v1:127.0.0.1:6813/3681890460] exists,up 8f5fb953-bba5-4b16-85f2-97c2129f3657 2026-02-17T11:29:01.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:29:01.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:29:01.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:29:01.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:125: TEST_allow_repair_during_recovery: create_rbd_pool 2026-02-17T11:29:01.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T11:29:01.709 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' does not exist 2026-02-17T11:29:01.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T11:29:01.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T11:29:01.929 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T11:29:01.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:29:02.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T11:29:03.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:126: TEST_allow_repair_during_recovery: wait_for_clean 2026-02-17T11:29:03.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:29:03.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:29:03.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:29:03.221 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:29:03.222 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:29:03.222 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:29:03.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:29:03.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:29:03.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:29:03.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:29:03.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:29:03.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:29:03.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:29:03.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:29:03.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:29:03.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:29:03.470 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:29:03.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:29:03.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:29:03.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:29:03.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T11:29:03.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T11:29:03.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T11:29:03.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:29:03.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:29:03.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672962 2026-02-17T11:29:03.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672962 2026-02-17T11:29:03.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-42949672962' 2026-02-17T11:29:03.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:29:03.617 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T11:29:03.617 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:29:03.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:29:03.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T11:29:03.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:29:03.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T11:29:03.620 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836484 2026-02-17T11:29:03.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T11:29:03.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:29:03.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T11:29:03.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:29:04.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:29:04.781 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:29:04.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T11:29:04.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:29:04.933 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672962 2026-02-17T11:29:04.933 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:29:04.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:29:04.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672962 2026-02-17T11:29:04.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:29:04.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672962 2026-02-17T11:29:04.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672962' 2026-02-17T11:29:04.938 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672962 2026-02-17T11:29:04.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:29:05.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672962 -lt 42949672962 2026-02-17T11:29:05.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:29:05.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:29:05.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:29:05.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T11:29:05.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:29:05.292 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:29:05.292 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:29:05.292 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:29:05.292 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:29:05.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:29:05.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:29:05.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T11:29:05.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:29:05.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:29:05.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:29:05.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T11:29:05.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:29:05.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:29:05.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:128: TEST_allow_repair_during_recovery: add_something td/osd-scrub-repair rbd 2026-02-17T11:29:05.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:29:05.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=rbd 2026-02-17T11:29:05.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T11:29:05.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:29:05.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:29:05.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:29:05.826 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:29:05.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:29:06.032 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:29:06.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:29:06.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:29:06.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool rbd put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T11:29:06.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:129: TEST_allow_repair_during_recovery: get_not_primary rbd SOMETHING 2026-02-17T11:29:06.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=rbd 2026-02-17T11:29:06.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:29:06.064 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary rbd SOMETHING 2026-02-17T11:29:06.064 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=rbd 2026-02-17T11:29:06.064 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:29:06.065 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map rbd SOMETHING 2026-02-17T11:29:06.065 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:29:06.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T11:29:06.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map rbd SOMETHING 2026-02-17T11:29:06.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T11:29:06.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:129: TEST_allow_repair_during_recovery: corrupt_and_repair_one td/osd-scrub-repair rbd 0 2026-02-17T11:29:06.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T11:29:06.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=rbd 2026-02-17T11:29:06.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=0 2026-02-17T11:29:06.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T11:29:06.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:29:06.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:29:06.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:29:06.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:29:06.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T11:29:06.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:29:06.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:29:06.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:29:06.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:29:06.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:29:06.374 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:29:06.374 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:29:06.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:29:06.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:29:06.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:29:07.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:29:07.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T11:29:07.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:29:07.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:29:07.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:29:07.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:29:07.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:29:07.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-17T11:29:08.325 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-17T11:29:08.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:29:08.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:29:08.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:29:08.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:29:08.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:29:08.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:29:08.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:29:08.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:29:08.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:29:08.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:29:08.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:29:08.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:29:08.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:29:08.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:29:08.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:29:08.865 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:29:08.865 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:29:08.865 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:29:08.866 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:29:08.866 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:29:08.866 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:29:08.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:29:08.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:29:08.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:29:08.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:29:08.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:29:08.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:29:08.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:29:08.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:29:08.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:29:08.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:29:08.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:29:08.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:29:08.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:29:08.868 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:29:08.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:29:08.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:29:08.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:29:08.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:29:08.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:29:08.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:29:08.887 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:08.885+0000 7f95cf3858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:08.888 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:08.885+0000 7f95cf3858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:08.889 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:08.885+0000 7f95cf3858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:09.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:29:09.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:29:09.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:29:09.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:29:09.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:29:09.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:29:09.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:29:09.016 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:29:09.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:29:09.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:29:09.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:29:10.108 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:10.105+0000 7f95cf3858c0 -1 Falling back to public interface 2026-02-17T11:29:10.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:29:10.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:29:10.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:29:10.173 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:29:10.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:29:10.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:29:10.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:29:11.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:29:11.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:29:11.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:29:11.329 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:29:11.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:29:11.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:29:11.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:29:12.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:29:12.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:29:12.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:29:12.486 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:29:12.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:29:12.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:29:12.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:29:13.104 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:13.101+0000 7f95cf3858c0 -1 osd.0 19 log_to_monitors true 2026-02-17T11:29:13.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:29:13.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:29:13.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:29:13.640 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:29:13.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:29:13.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:29:13.790 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 24 up_thru 24 down_at 20 last_clean_interval [5,19) [v2:127.0.0.1:6802/734023917,v1:127.0.0.1:6803/734023917] [v2:127.0.0.1:6804/734023917,v1:127.0.0.1:6805/734023917] exists,up 1ab256e5-e915-4e27-9fd9-e42ee275fb60 2026-02-17T11:29:13.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:29:13.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:29:13.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:29:13.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:29:13.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:29:13.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:29:13.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:29:13.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:29:13.791 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:29:13.791 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:29:13.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:29:13.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:29:13.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:29:13.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:29:13.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:29:13.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:29:13.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:29:13.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:29:13.893 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:29:14.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:29:14.047 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:29:14.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:29:14.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:29:14.047 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:29:14.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215106 2026-02-17T11:29:14.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215106 2026-02-17T11:29:14.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215106' 2026-02-17T11:29:14.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:29:14.118 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:29:14.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672965 2026-02-17T11:29:14.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672965 2026-02-17T11:29:14.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215106 1-42949672965' 2026-02-17T11:29:14.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:29:14.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-103079215106 2026-02-17T11:29:14.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:29:14.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:29:14.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-103079215106 2026-02-17T11:29:14.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:29:14.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215106 2026-02-17T11:29:14.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 103079215106' 2026-02-17T11:29:14.193 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 103079215106 2026-02-17T11:29:14.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:29:14.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 103079215106 2026-02-17T11:29:14.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:29:15.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:29:15.353 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:29:15.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 103079215106 2026-02-17T11:29:15.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:29:16.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:29:16.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:29:16.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215106 -lt 103079215106 2026-02-17T11:29:16.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:29:16.658 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672965 2026-02-17T11:29:16.658 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:29:16.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:29:16.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672965 2026-02-17T11:29:16.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:29:16.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672965 2026-02-17T11:29:16.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672965' 2026-02-17T11:29:16.662 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 42949672965 2026-02-17T11:29:16.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:29:16.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672965 -lt 42949672965 2026-02-17T11:29:16.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:29:16.828 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:29:16.828 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:29:17.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T11:29:17.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:29:17.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:29:17.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:29:17.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:29:17.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:29:17.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:29:17.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:29:17.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T11:29:17.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:29:17.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:29:17.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:29:17.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T11:29:17.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:29:17.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:29:17.367 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg rbd SOMETHING 2026-02-17T11:29:17.367 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=rbd 2026-02-17T11:29:17.367 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:29:17.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map rbd SOMETHING 2026-02-17T11:29:17.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:29:17.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=1.3 2026-02-17T11:29:17.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 1.3 2026-02-17T11:29:17.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.3 2026-02-17T11:29:17.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.3 2026-02-17T11:29:17.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:29:17.523 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:29:17.523 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:29:17.523 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:29:17.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:29:01.926842+0000 2026-02-17T11:29:17.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.3 2026-02-17T11:29:17.816 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 1.3 on osd.1 to repair 2026-02-17T11:29:17.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.3 2026-02-17T11:29:01.926842+0000 2026-02-17T11:29:17.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.3 2026-02-17T11:29:17.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:29:01.926842+0000 2026-02-17T11:29:17.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:29:17.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:29:17.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:29:17.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T11:29:17.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:29:17.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:29:17.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:29:17.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:29:17.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:29:01.926842+0000 '>' 2026-02-17T11:29:01.926842+0000 2026-02-17T11:29:17.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:29:18.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:29:18.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:29:18.980 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T11:29:18.980 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:29:18.980 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:29:18.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:29:18.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:29:19.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:29:01.926842+0000 '>' 2026-02-17T11:29:01.926842+0000 2026-02-17T11:29:19.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:29:20.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:29:20.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:29:20.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T11:29:20.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:29:20.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:29:20.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:29:20.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:29:20.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:29:01.926842+0000 '>' 2026-02-17T11:29:01.926842+0000 2026-02-17T11:29:20.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:29:21.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:29:21.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:29:21.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T11:29:21.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:29:21.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:29:21.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:29:21.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:29:21.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:29:01.926842+0000 '>' 2026-02-17T11:29:01.926842+0000 2026-02-17T11:29:21.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:29:22.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:29:22.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:29:22.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T11:29:22.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:29:22.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:29:22.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:29:22.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:29:22.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:29:18.086989+0000 '>' 2026-02-17T11:29:01.926842+0000 2026-02-17T11:29:22.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:29:22.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:29:22.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:29:22.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:29:22.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:29:22.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:29:22.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:29:22.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:29:22.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:29:22.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:29:22.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:29:22.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:29:22.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:29:22.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:29:22.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:29:22.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:29:22.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:29:23.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:29:23.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:29:23.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:29:23.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:29:23.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:29:23.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:29:23.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:29:23.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T11:29:24.229 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::scrub_1.3:head#, (61) No data available 2026-02-17T11:29:24.230 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:29:24.230 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:29:24.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:29:24.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:29:24.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:29:24.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:29:24.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:29:24.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:29:24.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:29:24.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:29:24.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:29:24.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:29:24.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:29:24.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:29:24.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:29:24.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:29:24.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:29:24.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:29:24.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:29:24.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:29:24.776 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:29:24.776 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:29:24.776 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:29:24.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:29:24.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:29:24.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:29:24.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:29:24.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:29:24.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:29:24.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:29:24.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:29:24.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:29:24.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:29:24.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:29:24.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:29:24.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:29:24.779 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:29:24.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:29:24.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:29:24.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:29:24.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:29:24.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:29:24.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:29:24.794 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:24.789+0000 7f8764d138c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:24.795 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:24.793+0000 7f8764d138c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:24.796 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:24.793+0000 7f8764d138c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:24.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:29:24.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:29:24.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:29:24.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:29:24.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:29:24.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:29:24.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:29:24.933 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:29:24.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:29:24.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:29:25.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:29:26.060 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:26.057+0000 7f8764d138c0 -1 Falling back to public interface 2026-02-17T11:29:26.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:29:26.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:29:26.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:29:26.086 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:29:26.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:29:26.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:29:26.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:29:27.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:29:27.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:29:27.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:29:27.234 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:29:27.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:29:27.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:29:27.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:29:28.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:29:28.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:29:28.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:29:28.382 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:29:28.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:29:28.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:29:28.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:29:29.081 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:29.077+0000 7f8764d138c0 -1 osd.1 26 log_to_monitors true 2026-02-17T11:29:29.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:29:29.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:29:29.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:29:29.531 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:29:29.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:29:29.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:29:29.677 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 30 up_thru 30 down_at 27 last_clean_interval [10,26) [v2:127.0.0.1:6810/3919348561,v1:127.0.0.1:6811/3919348561] [v2:127.0.0.1:6812/3919348561,v1:127.0.0.1:6813/3919348561] exists,up 8f5fb953-bba5-4b16-85f2-97c2129f3657 2026-02-17T11:29:29.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:29:29.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:29:29.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:29:29.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:29:29.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:29:29.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:29:29.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:29:29.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:29:29.678 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:29:29.678 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:29:29.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:29:29.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:29:29.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:29:29.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:29:29.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:29:29.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:29:29.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:29:29.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:29:29.780 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:29:29.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:29:29.927 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:29:29.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:29:29.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:29:29.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:29:29.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215110 2026-02-17T11:29:29.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215110 2026-02-17T11:29:29.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215110' 2026-02-17T11:29:29.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:29:29.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:29:30.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018882 2026-02-17T11:29:30.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018882 2026-02-17T11:29:30.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215110 1-128849018882' 2026-02-17T11:29:30.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:29:30.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-103079215110 2026-02-17T11:29:30.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:29:30.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:29:30.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-103079215110 2026-02-17T11:29:30.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:29:30.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215110 2026-02-17T11:29:30.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 103079215110' 2026-02-17T11:29:30.071 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 103079215110 2026-02-17T11:29:30.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:29:30.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215109 -lt 103079215110 2026-02-17T11:29:30.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:29:31.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:29:31.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:29:31.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215109 -lt 103079215110 2026-02-17T11:29:31.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:29:32.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:29:32.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:29:32.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215110 -lt 103079215110 2026-02-17T11:29:32.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:29:32.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-128849018882 2026-02-17T11:29:32.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:29:32.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:29:32.541 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-128849018882 2026-02-17T11:29:32.541 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:29:32.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018882 2026-02-17T11:29:32.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 128849018882' 2026-02-17T11:29:32.542 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 128849018882 2026-02-17T11:29:32.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:29:32.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018882 -lt 128849018882 2026-02-17T11:29:32.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:29:32.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:29:32.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:29:32.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T11:29:32.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:29:32.896 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:29:32.896 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:29:32.896 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:29:32.896 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:29:32.896 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:29:32.897 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:29:33.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T11:29:33.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:29:33.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:29:33.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:29:33.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T11:29:33.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:29:33.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:29:33.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool rbd get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:29:33.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:29:33.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T11:29:33.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:29:33.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:29:33.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:29:33.245 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:29:33.245 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:29:33.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:29:33.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:29:33.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:29:33.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:29:33.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:29:33.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:29:33.355 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:29:33.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:29:33.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:29:33.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:29:33.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:29:33.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:29:33.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:29:33.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:29:33.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:29:33.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:29:33.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:29:33.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:29:33.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:29:33.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:29:33.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:29:33.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:29:33.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:29:33.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T11:29:33.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T11:29:33.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T11:29:33.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T11:29:33.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:29:33.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:29:33.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:29:33.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:29:33.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:29:33.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:29:33.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:29:33.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:29:33.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:29:33.376 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:29:33.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:29:33.377 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:29:33.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:29:33.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:29:33.379 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:29:33.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:29:33.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:29:33.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:29:33.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:29:33.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:29:33.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:29:33.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:29:33.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:29:33.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:29:33.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:29:33.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:29:33.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:29:33.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:29:33.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T11:29:33.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:29:33.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:29:33.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:29:33.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T11:29:33.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:29:33.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T11:29:33.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:29:33.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T11:29:33.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_bluestore_basic td/osd-scrub-repair 2026-02-17T11:29:33.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:438: TEST_auto_repair_bluestore_basic: local dir=td/osd-scrub-repair 2026-02-17T11:29:33.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:439: TEST_auto_repair_bluestore_basic: cluster_conf=(['osds_num']='3' ['pgs_in_pool']='1' ['pool_name']='testpool' ['extras']=' --osd_scrub_auto_repair=true') 2026-02-17T11:29:33.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:439: TEST_auto_repair_bluestore_basic: local -A cluster_conf 2026-02-17T11:29:33.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:445: TEST_auto_repair_bluestore_basic: local extr_dbg=3 2026-02-17T11:29:33.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:446: TEST_auto_repair_bluestore_basic: standard_scrub_cluster td/osd-scrub-repair cluster_conf 2026-02-17T11:29:33.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:229: standard_scrub_cluster: local dir=td/osd-scrub-repair 2026-02-17T11:29:33.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:230: standard_scrub_cluster: local -n args=cluster_conf 2026-02-17T11:29:33.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:232: standard_scrub_cluster: local OSDS=3 2026-02-17T11:29:33.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:233: standard_scrub_cluster: local pg_num=1 2026-02-17T11:29:33.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:234: standard_scrub_cluster: local poolname=testpool 2026-02-17T11:29:33.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:235: standard_scrub_cluster: args['pool_name']=testpool 2026-02-17T11:29:33.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:236: standard_scrub_cluster: local 'extra_pars= --osd_scrub_auto_repair=true' 2026-02-17T11:29:33.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:237: standard_scrub_cluster: local debug_msg=dbg 2026-02-17T11:29:33.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:240: standard_scrub_cluster: local saved_echo_flag=x 2026-02-17T11:29:33.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:241: standard_scrub_cluster: set +x 2026-02-17T11:29:33.677 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 64ba4b0f-c9da-4780-9f4e-212f30395cf8 2026-02-17T11:29:33.775 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:29:33.801 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:33.797+0000 7f0c29a3c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:33.802 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:33.801+0000 7f0c29a3c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:33.803 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:33.801+0000 7f0c29a3c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:33.804 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:33.801+0000 7f0c29a3c8c0 -1 bdev(0x55d741487c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:29:33.804 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:33.801+0000 7f0c29a3c8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T11:29:36.408 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:29:36.602 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:29:36.621 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:36.617+0000 7fa00363b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:36.622 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:36.621+0000 7fa00363b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:36.623 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:36.621+0000 7fa00363b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:36.755 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:29:37.912 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:29:38.139 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:38.137+0000 7fa00363b8c0 -1 Falling back to public interface 2026-02-17T11:29:39.068 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:29:40.224 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:29:41.165 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:41.161+0000 7fa00363b8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:29:41.379 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:29:42.546 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:29:42.700 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3593694030,v1:127.0.0.1:6803/3593694030] [v2:127.0.0.1:6804/3593694030,v1:127.0.0.1:6805/3593694030] exists,up 64ba4b0f-c9da-4780-9f4e-212f30395cf8 2026-02-17T11:29:42.705 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 9af7182c-5c6a-45b4-8e42-72e59e7164a2 2026-02-17T11:29:42.872 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:29:42.898 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:42.897+0000 7fc192d998c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:42.900 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:42.897+0000 7fc192d998c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:42.901 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:42.897+0000 7fc192d998c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:42.901 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:42.897+0000 7fc192d998c0 -1 bdev(0x55a596e3fc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:29:42.902 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:42.901+0000 7fc192d998c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T11:29:45.180 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:29:45.370 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:29:45.387 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:45.385+0000 7f58ff93e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:45.388 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:45.385+0000 7f58ff93e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:45.389 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:45.385+0000 7f58ff93e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:45.528 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:29:46.678 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:29:46.911 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:46.909+0000 7f58ff93e8c0 -1 Falling back to public interface 2026-02-17T11:29:47.829 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:29:48.981 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:29:49.886 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:49.885+0000 7f58ff93e8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:29:50.130 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:29:51.285 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:29:51.434 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4045524976,v1:127.0.0.1:6811/4045524976] [v2:127.0.0.1:6812/4045524976,v1:127.0.0.1:6813/4045524976] exists,up 9af7182c-5c6a-45b4-8e42-72e59e7164a2 2026-02-17T11:29:51.439 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 57340dc7-6b76-4e2b-8a97-0d15b95ddfd8 2026-02-17T11:29:51.613 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:29:51.641 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:51.637+0000 7f458e8088c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:51.643 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:51.641+0000 7f458e8088c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:51.644 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:51.641+0000 7f458e8088c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:51.644 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:51.641+0000 7f458e8088c0 -1 bdev(0x55ec8b4b5c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T11:29:51.644 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:51.641+0000 7f458e8088c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T11:29:53.904 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T11:29:54.100 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T11:29:54.125 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:54.121+0000 7f5b84b328c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:54.127 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:54.125+0000 7f5b84b328c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:54.128 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:54.125+0000 7f5b84b328c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:29:54.251 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:29:55.406 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:29:55.627 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:55.625+0000 7f5b84b328c0 -1 Falling back to public interface 2026-02-17T11:29:56.561 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:29:57.717 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:29:58.649 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:29:58.645+0000 7f5b84b328c0 -1 osd.2 0 log_to_monitors true 2026-02-17T11:29:58.872 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:30:00.047 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:30:00.197 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2891846174,v1:127.0.0.1:6819/2891846174] [v2:127.0.0.1:6820/2891846174,v1:127.0.0.1:6821/2891846174] exists,up 57340dc7-6b76-4e2b-8a97-0d15b95ddfd8 2026-02-17T11:30:00.398 INFO:tasks.workunit.client.0.trial128.stderr:pool 'testpool' created 2026-02-17T11:30:01.880 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836485 2026-02-17T11:30:02.044 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705669 2026-02-17T11:30:02.213 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542146 2026-02-17T11:30:04.028 INFO:tasks.workunit.client.0.trial128.stdout:standard_scrub_cluster: dbg: test pool is testpool 1 2026-02-17T11:30:04.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:447: TEST_auto_repair_bluestore_basic: local poolid=1 2026-02-17T11:30:04.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:448: TEST_auto_repair_bluestore_basic: local poolname=testpool 2026-02-17T11:30:04.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:450: TEST_auto_repair_bluestore_basic: ceph osd pool set testpool size 2 2026-02-17T11:30:04.226 INFO:tasks.workunit.client.0.trial128.stderr:set pool 1 size to 2 2026-02-17T11:30:04.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:451: TEST_auto_repair_bluestore_basic: wait_for_clean 2026-02-17T11:30:04.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:30:04.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:30:04.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:30:04.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:30:04.238 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:30:04.238 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:30:04.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:30:04.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:30:04.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:30:04.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:30:04.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:30:04.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:30:04.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:30:04.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:30:04.353 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:30:04.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:30:04.506 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:30:04.506 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:30:04.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:30:04.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:30:04.507 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:30:04.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T11:30:04.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T11:30:04.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T11:30:04.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:30:04.576 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:30:04.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705670 2026-02-17T11:30:04.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705670 2026-02-17T11:30:04.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705670' 2026-02-17T11:30:04.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:30:04.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:30:04.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542147 2026-02-17T11:30:04.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542147 2026-02-17T11:30:04.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705670 2-60129542147' 2026-02-17T11:30:04.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:30:04.716 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T11:30:04.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:30:04.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:30:04.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T11:30:04.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:30:04.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T11:30:04.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T11:30:04.721 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836487 2026-02-17T11:30:04.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:30:04.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-17T11:30:04.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:30:05.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:30:05.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:30:06.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T11:30:06.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:30:06.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705670 2026-02-17T11:30:06.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:30:06.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:30:06.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705670 2026-02-17T11:30:06.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:30:06.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705670 2026-02-17T11:30:06.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-17T11:30:06.037 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705670 2026-02-17T11:30:06.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:30:06.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705670 -lt 38654705670 2026-02-17T11:30:06.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:30:06.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542147 2026-02-17T11:30:06.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:30:06.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:30:06.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542147 2026-02-17T11:30:06.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:30:06.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542147 2026-02-17T11:30:06.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-17T11:30:06.204 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542147 2026-02-17T11:30:06.204 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:30:06.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542148 -lt 60129542147 2026-02-17T11:30:06.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:30:06.361 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:06.361 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:06.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:30:06.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:30:06.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:30:06.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:30:06.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:30:06.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:30:06.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:30:06.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:30:06.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T11:30:06.691 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:30:06.691 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:06.691 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:06.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T11:30:06.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T11:30:06.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T11:30:06.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T11:30:06.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:30:06.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T11:30:06.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:30:06.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:30:06.972 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:30:06.972 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:30:06.972 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:30:06.972 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:30:06.972 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:30:06.972 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:30:07.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T11:30:07.118 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:30:07.118 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:07.118 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:07.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T11:30:07.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T11:30:07.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:30:07.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:30:07.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:30:07.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:30:07.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:30:07.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:30:07.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:30:07.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:30:07.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:30:07.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T11:30:07.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:30:07.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T11:30:07.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:30:07.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:30:07.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:30:07.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:30:07.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:30:07.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:30:07.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:30:07.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:30:07.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T11:30:07.828 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:30:07.828 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:07.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:08.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T11:30:08.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T11:30:08.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:30:08.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:30:08.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:30:08.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:30:08.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:30:08.008 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:30:08.008 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:30:08.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:30:08.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:30:08.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T11:30:08.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:30:08.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T11:30:08.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:30:08.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:30:08.587 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:30:08.587 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:30:08.587 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:30:08.587 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:30:08.588 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:30:08.588 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:30:08.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:30:08.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:30:08.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:08.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:08.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:30:08.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:30:08.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:30:08.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:454: TEST_auto_repair_bluestore_basic: local payload=ABCDEF 2026-02-17T11:30:08.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:455: TEST_auto_repair_bluestore_basic: echo ABCDEF 2026-02-17T11:30:08.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:456: TEST_auto_repair_bluestore_basic: rados --pool testpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T11:30:08.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:460: TEST_auto_repair_bluestore_basic: get_not_primary testpool SOMETHING 2026-02-17T11:30:08.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T11:30:08.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:30:08.944 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T11:30:08.944 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:30:08.944 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:30:08.944 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:30:08.944 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:30:09.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T11:30:09.099 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:30:09.099 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T11:30:09.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:460: TEST_auto_repair_bluestore_basic: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T11:30:09.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:30:09.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:30:09.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:30:09.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:30:09.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T11:30:09.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:30:09.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:30:09.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:30:09.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:30:09.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:30:09.258 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:30:09.258 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:30:09.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:30:09.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:30:09.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:30:10.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:30:10.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T11:30:10.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:30:10.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:30:10.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:30:10.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:30:10.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:30:10.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-17T11:30:11.214 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-17T11:30:11.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:30:11.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:30:11.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:30:11.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:30:11.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:30:11.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:30:11.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:30:11.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:30:11.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:30:11.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:30:11.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:30:11.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:30:11.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:30:11.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:30:11.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:30:11.752 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:30:11.752 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:30:11.752 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:30:11.752 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:30:11.752 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:30:11.752 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:30:11.752 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:30:11.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:30:11.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:30:11.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:30:11.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:30:11.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:30:11.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:30:11.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:30:11.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:30:11.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:30:11.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:30:11.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:30:11.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:30:11.754 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:30:11.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:30:11.754 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:30:11.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:30:11.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:30:11.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:30:11.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:30:11.772 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:11.769+0000 7f416fa238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:11.774 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:11.773+0000 7f416fa238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:11.775 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:11.773+0000 7f416fa238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:11.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:30:11.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:30:11.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:30:11.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:30:11.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:30:11.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:11.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:30:11.911 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:30:11.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:11.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:12.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:13.035 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:13.033+0000 7f416fa238c0 -1 Falling back to public interface 2026-02-17T11:30:13.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:13.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:13.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:30:13.062 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:30:13.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:13.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:13.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:14.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:14.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:14.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:30:14.217 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:30:14.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:14.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:14.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:15.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:15.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:15.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:30:15.369 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:30:15.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:15.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:15.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:16.305 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:16.301+0000 7f416fa238c0 -1 osd.0 19 log_to_monitors true 2026-02-17T11:30:16.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:16.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:16.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:30:16.518 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:30:16.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:16.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:16.667 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 23 up_thru 0 down_at 20 last_clean_interval [5,19) [v2:127.0.0.1:6802/3759894948,v1:127.0.0.1:6803/3759894948] [v2:127.0.0.1:6804/3759894948,v1:127.0.0.1:6805/3759894948] exists,up 64ba4b0f-c9da-4780-9f4e-212f30395cf8 2026-02-17T11:30:16.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:30:16.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:30:16.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:30:16.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:30:16.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:30:16.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:30:16.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:30:16.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:30:16.669 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:30:16.669 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:30:16.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:30:16.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:30:16.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:30:16.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:30:16.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:30:16.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:30:16.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:30:16.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:30:16.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:30:16.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:30:16.937 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:30:16.937 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:30:16.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:30:16.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:30:16.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:30:17.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247810 2026-02-17T11:30:17.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247810 2026-02-17T11:30:17.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810' 2026-02-17T11:30:17.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:30:17.007 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:30:17.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705674 2026-02-17T11:30:17.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705674 2026-02-17T11:30:17.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810 1-38654705674' 2026-02-17T11:30:17.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:30:17.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:30:17.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542151 2026-02-17T11:30:17.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542151 2026-02-17T11:30:17.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810 1-38654705674 2-60129542151' 2026-02-17T11:30:17.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:30:17.149 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247810 2026-02-17T11:30:17.149 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:30:17.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:30:17.151 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247810 2026-02-17T11:30:17.151 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:30:17.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247810 2026-02-17T11:30:17.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247810' 2026-02-17T11:30:17.152 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 98784247810 2026-02-17T11:30:17.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:30:17.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 98784247810 2026-02-17T11:30:17.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:30:18.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:30:18.315 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:30:18.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247810 -lt 98784247810 2026-02-17T11:30:18.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:30:18.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705674 2026-02-17T11:30:18.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:30:18.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:30:18.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705674 2026-02-17T11:30:18.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:30:18.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705674 2026-02-17T11:30:18.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-17T11:30:18.475 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705674 2026-02-17T11:30:18.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:30:18.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705674 -lt 38654705674 2026-02-17T11:30:18.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:30:18.635 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542151 2026-02-17T11:30:18.635 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:30:18.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:30:18.637 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542151 2026-02-17T11:30:18.637 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:30:18.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542151 2026-02-17T11:30:18.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542151' 2026-02-17T11:30:18.638 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542151 2026-02-17T11:30:18.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:30:18.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542151 -lt 60129542151 2026-02-17T11:30:18.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:30:18.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:18.798 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:18.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:30:18.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:30:18.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:30:18.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:30:18.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:30:18.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:30:18.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:30:18.986 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:30:19.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:30:19.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:30:19.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:19.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:19.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:30:19.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:30:19.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:30:19.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:461: TEST_auto_repair_bluestore_basic: ceph tell 'osd.*' config set osd_scrub_auto_repair true 2026-02-17T11:30:19.388 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T11:30:19.388 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_auto_repair = '' (not observed, change may require restart) osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-17T11:30:19.389 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:30:19.395 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T11:30:19.395 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_auto_repair = '' (not observed, change may require restart) " 2026-02-17T11:30:19.395 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:30:19.401 INFO:tasks.workunit.client.0.trial128.stdout:osd.2: { 2026-02-17T11:30:19.401 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_auto_repair = '' (not observed, change may require restart) " 2026-02-17T11:30:19.401 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:30:19.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:463: TEST_auto_repair_bluestore_basic: get_pg testpool SOMETHING 2026-02-17T11:30:19.409 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=testpool 2026-02-17T11:30:19.409 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:30:19.409 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map testpool SOMETHING 2026-02-17T11:30:19.409 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:30:19.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:463: TEST_auto_repair_bluestore_basic: local pgid=1.0 2026-02-17T11:30:19.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:464: TEST_auto_repair_bluestore_basic: get_primary testpool SOMETHING 2026-02-17T11:30:19.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:30:19.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:30:19.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:30:19.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:30:19.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:464: TEST_auto_repair_bluestore_basic: local primary=1 2026-02-17T11:30:19.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:465: TEST_auto_repair_bluestore_basic: get_last_scrub_stamp 1.0 2026-02-17T11:30:19.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:30:19.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:30:19.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:30:19.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:30:19.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:465: TEST_auto_repair_bluestore_basic: local last_scrub_stamp=2026-02-17T11:30:00.397560+0000 2026-02-17T11:30:19.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:468: TEST_auto_repair_bluestore_basic: ceph tell 1.0 schedule-deep-scrub 2026-02-17T11:30:19.929 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T11:30:19.929 INFO:tasks.workunit.client.0.trial128.stdout: "deep": true, 2026-02-17T11:30:19.930 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T11:30:19.930 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-03T11:28:39.929938+0000" 2026-02-17T11:30:19.930 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:30:19.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:471: TEST_auto_repair_bluestore_basic: wait_for_scrub 1.0 2026-02-17T11:30:00.397560+0000 2026-02-17T11:30:19.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T11:30:19.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:30:00.397560+0000 2026-02-17T11:30:19.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:30:19.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:30:19.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:30:19.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:30:19.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:30:19.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:30:19.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:30:19.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:30:20.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:30:00.397560+0000 '>' 2026-02-17T11:30:00.397560+0000 2026-02-17T11:30:20.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:30:21.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:30:21.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:30:21.089 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:30:21.090 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:30:21.090 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:30:21.090 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:30:21.090 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:30:21.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:30:00.397560+0000 '>' 2026-02-17T11:30:00.397560+0000 2026-02-17T11:30:21.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:30:22.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:30:22.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:30:22.241 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:30:22.241 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:30:22.241 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:30:22.241 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:30:22.241 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:30:22.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:30:20.094443+0000 '>' 2026-02-17T11:30:00.397560+0000 2026-02-17T11:30:22.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:30:22.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:472: TEST_auto_repair_bluestore_basic: wait_for_clean 2026-02-17T11:30:22.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:30:22.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:30:22.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:30:22.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:30:22.389 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:30:22.389 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:30:22.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:30:22.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:30:22.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:30:22.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:30:22.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:30:22.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:30:22.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:30:22.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:30:22.494 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:30:22.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:30:22.652 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:30:22.652 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:30:22.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:30:22.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:30:22.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:30:22.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247812 2026-02-17T11:30:22.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247812 2026-02-17T11:30:22.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247812' 2026-02-17T11:30:22.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:30:22.722 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:30:22.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705676 2026-02-17T11:30:22.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705676 2026-02-17T11:30:22.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247812 1-38654705676' 2026-02-17T11:30:22.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:30:22.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:30:22.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542153 2026-02-17T11:30:22.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542153 2026-02-17T11:30:22.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247812 1-38654705676 2-60129542153' 2026-02-17T11:30:22.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:30:22.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247812 2026-02-17T11:30:22.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:30:22.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:30:22.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247812 2026-02-17T11:30:22.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:30:22.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247812 2026-02-17T11:30:22.876 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 98784247812 2026-02-17T11:30:22.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247812' 2026-02-17T11:30:22.877 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:30:23.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247811 -lt 98784247812 2026-02-17T11:30:23.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:30:24.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:30:24.027 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:30:24.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247812 -lt 98784247812 2026-02-17T11:30:24.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:30:24.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705676 2026-02-17T11:30:24.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:30:24.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:30:24.188 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705676 2026-02-17T11:30:24.188 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:30:24.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705676 2026-02-17T11:30:24.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705676' 2026-02-17T11:30:24.189 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705676 2026-02-17T11:30:24.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:30:24.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705676 -lt 38654705676 2026-02-17T11:30:24.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:30:24.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542153 2026-02-17T11:30:24.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:30:24.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:30:24.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542153 2026-02-17T11:30:24.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:30:24.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542153 2026-02-17T11:30:24.348 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542153 2026-02-17T11:30:24.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-17T11:30:24.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:30:24.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-17T11:30:24.512 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:30:24.512 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:24.512 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:24.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:30:24.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:30:24.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:30:24.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:30:24.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:30:24.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:30:24.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:30:24.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:30:24.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:30:24.848 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:30:24.849 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:24.849 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:25.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:30:25.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:30:25.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:30:25.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:473: TEST_auto_repair_bluestore_basic: ceph pg dump pgs 2026-02-17T11:30:25.169 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T11:30:25.169 INFO:tasks.workunit.client.0.trial128.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean 2026-02-17T11:30:20.096426+0000 19'1 25:59 [1,0] 1 [1,0] 1 19'1 2026-02-17T11:30:20.094443+0000 19'1 2026-02-17T11:30:20.094443+0000 0 1 periodic scrub scheduled @ 2026-02-18T11:30:20.094443+0000 1 0 2026-02-17T11:30:25.169 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:30:25.169 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T11:30:25.170 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:30:25.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:476: TEST_auto_repair_bluestore_basic: get_not_primary testpool SOMETHING 2026-02-17T11:30:25.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T11:30:25.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:30:25.179 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T11:30:25.179 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:30:25.179 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:30:25.179 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:30:25.179 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:30:25.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T11:30:25.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:30:25.329 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T11:30:25.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:476: TEST_auto_repair_bluestore_basic: objectstore_tool td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T11:30:25.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:30:25.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:30:25.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:30:25.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:30:25.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T11:30:25.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:30:25.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:30:25.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:30:25.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:30:25.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:30:25.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:30:25.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:30:25.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:30:25.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:30:25.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:30:26.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:30:26.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T11:30:26.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:30:26.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:30:26.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:30:26.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:30:26.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:30:26.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING list-attrs 2026-02-17T11:30:27.104 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:30:27.105 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:30:27.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:30:27.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:30:27.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:30:27.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:30:27.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:30:27.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:30:27.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:30:27.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:30:27.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:30:27.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:30:27.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:30:27.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:30:27.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:30:27.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:30:27.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:30:27.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:30:27.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:30:27.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:30:27.640 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:30:27.640 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:30:27.640 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:30:27.640 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:30:27.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:30:27.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:30:27.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:30:27.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:30:27.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:30:27.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:30:27.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:30:27.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:30:27.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:30:27.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:30:27.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:30:27.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:30:27.642 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:30:27.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:30:27.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:30:27.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:30:27.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:30:27.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:30:27.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:30:27.660 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:27.657+0000 7fdffa25d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:27.662 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:27.661+0000 7fdffa25d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:27.662 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:27.661+0000 7fdffa25d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:27.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:30:27.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:30:27.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:30:27.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:30:27.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:30:27.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:27.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:30:27.795 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:30:27.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:27.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:27.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:28.911 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:28.909+0000 7fdffa25d8c0 -1 Falling back to public interface 2026-02-17T11:30:28.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:28.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:28.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:30:28.951 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:30:28.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:28.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:29.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:30.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:30.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:30.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:30:30.105 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:30:30.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:30.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:30.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:31.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:31.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:31.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:30:31.260 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:30:31.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:31.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:31.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:31.937 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:31.933+0000 7fdffa25d8c0 -1 osd.0 25 log_to_monitors true 2026-02-17T11:30:32.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:32.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:32.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:30:32.416 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:30:32.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:32.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:32.569 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 29 up_thru 0 down_at 26 last_clean_interval [23,25) [v2:127.0.0.1:6802/2557985918,v1:127.0.0.1:6803/2557985918] [v2:127.0.0.1:6804/2557985918,v1:127.0.0.1:6805/2557985918] exists,up 64ba4b0f-c9da-4780-9f4e-212f30395cf8 2026-02-17T11:30:32.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:30:32.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:30:32.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:30:32.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:30:32.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:30:32.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:30:32.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:30:32.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:30:32.571 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:30:32.571 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:30:32.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:30:32.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:30:32.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:30:32.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:30:32.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:30:32.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:30:32.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:30:32.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:30:32.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:30:32.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:30:32.820 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:30:32.820 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:30:32.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:30:32.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:30:32.820 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:30:32.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051586 2026-02-17T11:30:32.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051586 2026-02-17T11:30:32.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586' 2026-02-17T11:30:32.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:30:32.893 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:30:32.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705679 2026-02-17T11:30:32.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705679 2026-02-17T11:30:32.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586 1-38654705679' 2026-02-17T11:30:32.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:30:32.964 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:30:33.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542156 2026-02-17T11:30:33.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542156 2026-02-17T11:30:33.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586 1-38654705679 2-60129542156' 2026-02-17T11:30:33.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:30:33.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-124554051586 2026-02-17T11:30:33.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:30:33.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:30:33.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-124554051586 2026-02-17T11:30:33.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:30:33.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051586 2026-02-17T11:30:33.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 124554051586' 2026-02-17T11:30:33.040 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 124554051586 2026-02-17T11:30:33.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:30:33.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247812 -lt 124554051586 2026-02-17T11:30:33.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:30:34.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:30:34.201 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:30:34.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051586 -lt 124554051586 2026-02-17T11:30:34.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:30:34.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705679 2026-02-17T11:30:34.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:30:34.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:30:34.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705679 2026-02-17T11:30:34.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:30:34.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705679 2026-02-17T11:30:34.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705679' 2026-02-17T11:30:34.361 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705679 2026-02-17T11:30:34.362 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:30:34.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705679 -lt 38654705679 2026-02-17T11:30:34.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:30:34.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542156 2026-02-17T11:30:34.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:30:34.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:30:34.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542156 2026-02-17T11:30:34.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:30:34.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542156 2026-02-17T11:30:34.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542156' 2026-02-17T11:30:34.519 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542156 2026-02-17T11:30:34.519 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:30:34.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542156 -lt 60129542156 2026-02-17T11:30:34.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:30:34.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:34.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:34.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:30:34.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:30:34.862 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:30:34.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:30:34.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:30:34.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:30:34.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:30:34.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:30:35.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T11:30:35.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:30:35.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:35.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:35.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T11:30:35.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T11:30:35.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T11:30:35.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T11:30:35.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:30:35.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T11:30:35.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:30:35.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:30:35.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:30:35.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:30:35.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:30:35.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:30:35.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:30:35.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:30:35.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T11:30:35.460 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:30:35.461 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:35.461 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:35.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T11:30:35.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T11:30:35.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:30:35.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:30:35.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:30:35.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:30:35.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:30:35.647 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:30:35.647 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:30:35.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:30:35.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:30:35.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T11:30:35.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:30:35.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T11:30:36.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:30:36.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:30:36.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:30:36.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:30:36.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:30:36.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:30:36.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:30:36.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:30:36.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T11:30:36.188 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:30:36.189 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:36.189 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T11:30:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T11:30:36.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:30:36.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:30:36.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:30:36.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:30:36.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:30:36.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:30:36.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:30:36.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:30:36.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:30:36.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T11:30:36.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:30:36.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T11:30:36.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:30:36.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:30:36.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:30:36.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:30:36.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:30:36.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:30:36.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:30:36.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:30:37.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T11:30:37.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:30:37.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:37.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:37.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T11:30:37.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T11:30:37.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:30:37.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:30:37.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:30:37.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:30:37.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:30:37.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:30:37.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:30:37.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:30:37.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:30:37.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 13 )) 2026-02-17T11:30:37.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:30:37.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T11:30:38.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:30:38.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:30:38.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:30:38.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:30:38.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:30:38.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:30:38.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:30:38.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:30:38.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:30:38.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:30:38.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:38.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:38.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:30:38.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:30:38.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:30:38.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:477: TEST_auto_repair_bluestore_basic: get_not_primary testpool SOMETHING 2026-02-17T11:30:38.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T11:30:38.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:30:38.615 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T11:30:38.615 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:30:38.615 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:30:38.615 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:30:38.615 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:30:38.767 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T11:30:38.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:30:38.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T11:30:38.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:477: TEST_auto_repair_bluestore_basic: objectstore_tool td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-17T11:30:38.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:30:38.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:30:38.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:30:38.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:30:38.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-17T11:30:38.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:30:38.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:30:38.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:30:38.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:30:38.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:30:38.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:30:38.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:30:38.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:30:38.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:30:38.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:30:40.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:30:40.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-17T11:30:40.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:30:40.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:30:40.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:30:40.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:30:40.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:30:40.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-17T11:30:41.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:30:41.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:30:41.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:30:41.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:30:41.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:30:41.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:30:41.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:30:41.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:30:41.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:30:41.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:30:41.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:30:41.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:30:41.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:30:41.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:30:41.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:30:41.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:30:41.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:30:41.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:30:41.096 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:30:41.096 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:30:41.096 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:30:41.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:30:41.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:30:41.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:30:41.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:30:41.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:30:41.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:30:41.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:30:41.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:30:41.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:30:41.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:30:41.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:30:41.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:30:41.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:30:41.098 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:30:41.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:30:41.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:30:41.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:30:41.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:30:41.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:30:41.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:30:41.114 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:41.113+0000 7fcc383a48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:41.115 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:41.113+0000 7fcc383a48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:41.116 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:41.113+0000 7fcc383a48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:41.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:30:41.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:30:41.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:30:41.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:30:41.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:30:41.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:41.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:30:41.250 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:30:41.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:41.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:41.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:42.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:42.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:42.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:30:42.407 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:30:42.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:42.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:42.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:42.603 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:42.601+0000 7fcc383a48c0 -1 Falling back to public interface 2026-02-17T11:30:43.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:43.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:43.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:30:43.561 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:30:43.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:43.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:43.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:44.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:44.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:44.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:30:44.715 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:30:44.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:44.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:44.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:45.576 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:45.573+0000 7fcc383a48c0 -1 osd.0 30 log_to_monitors true 2026-02-17T11:30:45.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:45.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:45.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:30:45.874 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:30:45.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:45.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:46.028 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 34 up_thru 0 down_at 31 last_clean_interval [29,30) [v2:127.0.0.1:6802/2133657458,v1:127.0.0.1:6803/2133657458] [v2:127.0.0.1:6804/2133657458,v1:127.0.0.1:6805/2133657458] exists,up 64ba4b0f-c9da-4780-9f4e-212f30395cf8 2026-02-17T11:30:46.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:30:46.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:30:46.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:30:46.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:30:46.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:30:46.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:30:46.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:30:46.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:30:46.029 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:30:46.029 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:30:46.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:30:46.030 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:30:46.030 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:30:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:30:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:30:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:30:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:30:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:30:46.139 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:30:46.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:30:46.295 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:30:46.295 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:30:46.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:30:46.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:30:46.295 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:30:46.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888066 2026-02-17T11:30:46.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888066 2026-02-17T11:30:46.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888066' 2026-02-17T11:30:46.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:30:46.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:30:46.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705683 2026-02-17T11:30:46.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705683 2026-02-17T11:30:46.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888066 1-38654705683' 2026-02-17T11:30:46.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:30:46.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:30:46.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542160 2026-02-17T11:30:46.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542160 2026-02-17T11:30:46.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888066 1-38654705683 2-60129542160' 2026-02-17T11:30:46.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:30:46.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-146028888066 2026-02-17T11:30:46.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:30:46.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:30:46.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-146028888066 2026-02-17T11:30:46.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:30:46.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888066 2026-02-17T11:30:46.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 146028888066' 2026-02-17T11:30:46.516 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 146028888066 2026-02-17T11:30:46.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:30:46.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 146028888066 2026-02-17T11:30:46.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:30:47.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:30:47.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:30:47.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 146028888066 2026-02-17T11:30:47.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:30:48.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:30:48.834 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:30:48.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888066 -lt 146028888066 2026-02-17T11:30:48.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:30:48.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705683 2026-02-17T11:30:48.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:30:48.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:30:48.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705683 2026-02-17T11:30:48.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:30:48.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705683 2026-02-17T11:30:48.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705683' 2026-02-17T11:30:48.997 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705683 2026-02-17T11:30:48.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:30:49.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705683 -lt 38654705683 2026-02-17T11:30:49.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:30:49.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542160 2026-02-17T11:30:49.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:30:49.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:30:49.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542160 2026-02-17T11:30:49.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:30:49.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542160 2026-02-17T11:30:49.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542160' 2026-02-17T11:30:49.155 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542160 2026-02-17T11:30:49.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:30:49.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542160 -lt 60129542160 2026-02-17T11:30:49.321 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:30:49.322 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:49.322 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:49.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:30:49.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:30:49.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:30:49.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:30:49.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:30:49.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:30:49.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:30:49.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:30:49.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:30:49.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:30:49.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:30:49.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:30:49.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:30:49.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:30:49.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:30:49.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:478: TEST_auto_repair_bluestore_basic: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:30:49.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:479: TEST_auto_repair_bluestore_basic: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-17T11:30:49.853 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:30:20.093+0000 7f58dd8bc640 10 osd.1 pg_epoch: 25 pg[1.0( v 19'1 (0'0,19'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=19'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] planned AUTO_REPAIR TIME_FOR_DEEP] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T11:30:49.853 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:30:20.093+0000 7f58dd8bc640 10 osd.1 pg_epoch: 25 pg[1.0( v 19'1 (0'0,19'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=19'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-17T11:30:49.853 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:30:20.093+0000 7f58dd8bc640 15 osd.1 pg_epoch: 25 pg[1.0( v 19'1 (0'0,19'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=19'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish: 1 errors. 1 errors fixed 2026-02-17T11:30:49.853 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:30:20.093+0000 7f58dd8bc640 20 osd.1 pg_epoch: 25 pg[1.0( v 19'1 (0'0,19'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=19'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish All may be fixed 2026-02-17T11:30:49.853 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:30:20.093+0000 7f58dd8bc640 19 osd.1 pg_epoch: 25 pg[1.0( v 19'1 (0'0,19'1] local-lis/les=23/24 n=1 ec=15/15 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=19'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T11:30:49.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T11:30:49.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:30:49.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:30:49.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:30:49.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:30:49.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:30:49.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:30:49.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:30:49.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:30:49.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:30:49.964 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:30:49.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:30:49.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:30:49.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:30:49.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:30:49.967 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:30:49.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:30:49.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:30:49.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:30:49.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:30:49.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:30:49.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:30:49.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:30:49.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:30:49.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:30:49.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:30:49.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:30:49.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:30:49.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:30:49.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T11:30:49.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T11:30:49.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T11:30:49.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T11:30:49.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:30:49.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:30:49.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:30:49.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:30:49.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:30:49.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:30:49.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:30:49.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:30:49.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:30:49.990 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:30:49.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:30:49.991 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:30:49.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:30:49.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:30:49.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:30:49.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:30:49.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:30:49.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:30:49.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:30:49.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:30:49.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:30:49.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:30:49.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:30:49.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:30:49.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:30:49.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:30:49.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:30:49.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:30:49.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T11:30:50.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:30:50.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:30:50.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:30:50.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T11:30:50.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:30:50.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T11:30:50.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:30:50.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T11:30:50.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_bluestore_failed td/osd-scrub-repair 2026-02-17T11:30:50.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:539: TEST_auto_repair_bluestore_failed: local dir=td/osd-scrub-repair 2026-02-17T11:30:50.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:540: TEST_auto_repair_bluestore_failed: local poolname=testpool 2026-02-17T11:30:50.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:543: TEST_auto_repair_bluestore_failed: run_mon td/osd-scrub-repair a 2026-02-17T11:30:50.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T11:30:50.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T11:30:50.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T11:30:50.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T11:30:50.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T11:30:50.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T11:30:50.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T11:30:50.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:30:50.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:30:50.025 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:30:50.025 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:30:50.025 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:30:50.025 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:30:50.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T11:30:50.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T11:30:50.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T11:30:50.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:30:50.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:30:50.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T11:30:50.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T11:30:50.050 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:30:50.050 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:30:50.051 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:30:50.051 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:30:50.051 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:30:50.051 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:30:50.051 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:30:50.052 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:30:50.052 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T11:30:50.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T11:30:50.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:30:50.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:30:50.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T11:30:50.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T11:30:50.106 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:30:50.106 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:30:50.106 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:30:50.106 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:30:50.106 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:30:50.106 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:30:50.107 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:30:50.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:30:50.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T11:30:50.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:544: TEST_auto_repair_bluestore_failed: run_mgr td/osd-scrub-repair x 2026-02-17T11:30:50.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T11:30:50.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T11:30:50.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T11:30:50.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T11:30:50.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T11:30:50.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T11:30:50.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T11:30:50.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:30:50.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:30:50.254 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:30:50.254 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:30:50.254 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:30:50.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:30:50.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:30:50.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:30:50.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:547: TEST_auto_repair_bluestore_failed: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:30:50.272 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:548: TEST_auto_repair_bluestore_failed: seq 0 2 2026-02-17T11:30:50.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:548: TEST_auto_repair_bluestore_failed: for id in $(seq 0 2) 2026-02-17T11:30:50.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:549: TEST_auto_repair_bluestore_failed: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:30:50.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:30:50.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:30:50.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T11:30:50.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:30:50.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:30:50.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:30:50.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:30:50.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:30:50.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:30:50.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:30:50.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:30:50.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:30:50.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:30:50.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:30:50.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:30:50.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:30:50.278 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:30:50.278 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:30:50.278 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:30:50.278 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:30:50.278 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:30:50.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:30:50.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:30:50.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:30:50.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:30:50.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:30:50.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:30:50.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:30:50.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:30:50.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:30:50.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:30:50.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:30:50.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:30:50.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:30:50.280 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:30:50.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=efe9f3a2-7c61-4d13-8795-ddce616dc10e 2026-02-17T11:30:50.280 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 efe9f3a2-7c61-4d13-8795-ddce616dc10e 2026-02-17T11:30:50.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 efe9f3a2-7c61-4d13-8795-ddce616dc10e' 2026-02-17T11:30:50.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:30:50.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBqUZRpmDMxERAAYiUhg46S9dWv90UjvFVgrw== 2026-02-17T11:30:50.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBqUZRpmDMxERAAYiUhg46S9dWv90UjvFVgrw=="}' 2026-02-17T11:30:50.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new efe9f3a2-7c61-4d13-8795-ddce616dc10e -i td/osd-scrub-repair/0/new.json 2026-02-17T11:30:50.378 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:30:50.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T11:30:50.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQBqUZRpmDMxERAAYiUhg46S9dWv90UjvFVgrw== --osd-uuid efe9f3a2-7c61-4d13-8795-ddce616dc10e 2026-02-17T11:30:50.407 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:50.404+0000 7fdcfe9748c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:50.408 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:50.404+0000 7fdcfe9748c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:50.409 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:50.408+0000 7fdcfe9748c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:50.410 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:50.408+0000 7fdcfe9748c0 -1 bdev(0x55a812cffc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:30:50.410 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:50.408+0000 7fdcfe9748c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T11:30:53.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T11:30:53.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:30:53.465 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:30:53.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T11:30:53.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:30:53.656 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:30:53.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T11:30:53.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:30:53.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:30:53.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:30:53.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:30:53.682 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:53.680+0000 7faae3ce28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:53.683 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:53.680+0000 7faae3ce28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:53.685 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:53.680+0000 7faae3ce28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:53.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T11:30:53.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:30:53.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:30:53.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:30:53.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:30:53.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:53.816 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:30:53.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:30:53.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:53.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:53.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:54.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:54.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:54.969 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:30:54.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:30:54.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:54.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:55.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:55.171 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:55.168+0000 7faae3ce28c0 -1 Falling back to public interface 2026-02-17T11:30:56.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:56.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:56.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:30:56.120 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:30:56.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:56.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:56.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:57.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:57.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:57.272 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:30:57.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:30:57.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:57.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:57.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:58.177 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:58.172+0000 7faae3ce28c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:30:58.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:58.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:58.428 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:30:58.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:30:58.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:58.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:58.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:30:59.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:30:59.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:30:59.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:30:59.581 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:30:59.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:30:59.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:30:59.728 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2306833256,v1:127.0.0.1:6803/2306833256] [v2:127.0.0.1:6804/2306833256,v1:127.0.0.1:6805/2306833256] exists,up efe9f3a2-7c61-4d13-8795-ddce616dc10e 2026-02-17T11:30:59.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:30:59.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:30:59.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:30:59.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:548: TEST_auto_repair_bluestore_failed: for id in $(seq 0 2) 2026-02-17T11:30:59.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:549: TEST_auto_repair_bluestore_failed: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:30:59.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:30:59.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:30:59.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T11:30:59.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:30:59.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:30:59.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:30:59.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:30:59.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:30:59.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:30:59.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:30:59.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:30:59.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:30:59.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:30:59.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:30:59.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:30:59.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:30:59.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:30:59.733 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:30:59.733 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:30:59.733 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:30:59.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:30:59.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:30:59.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:30:59.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:30:59.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:30:59.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:30:59.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:30:59.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:30:59.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:30:59.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:30:59.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:30:59.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:30:59.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:30:59.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:30:59.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:30:59.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=622dcc91-6b17-44f5-b56b-782afbf51e4a 2026-02-17T11:30:59.735 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 622dcc91-6b17-44f5-b56b-782afbf51e4a 2026-02-17T11:30:59.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 622dcc91-6b17-44f5-b56b-782afbf51e4a' 2026-02-17T11:30:59.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:30:59.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBzUZRpaABhLBAAdC2z0r22sqz/bBA+DrjRlA== 2026-02-17T11:30:59.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBzUZRpaABhLBAAdC2z0r22sqz/bBA+DrjRlA=="}' 2026-02-17T11:30:59.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 622dcc91-6b17-44f5-b56b-782afbf51e4a -i td/osd-scrub-repair/1/new.json 2026-02-17T11:30:59.898 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:30:59.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T11:30:59.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQBzUZRpaABhLBAAdC2z0r22sqz/bBA+DrjRlA== --osd-uuid 622dcc91-6b17-44f5-b56b-782afbf51e4a 2026-02-17T11:30:59.924 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:59.920+0000 7f76ec6318c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:59.925 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:59.920+0000 7f76ec6318c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:59.926 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:59.924+0000 7f76ec6318c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:30:59.927 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:59.924+0000 7f76ec6318c0 -1 bdev(0x55c912d25c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:30:59.927 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:30:59.924+0000 7f76ec6318c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T11:31:02.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T11:31:02.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:31:02.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T11:31:02.260 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:31:02.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:31:02.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T11:31:02.453 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:31:02.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:31:02.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:31:02.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:31:02.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:31:02.477 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:02.472+0000 7f4cd72d88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:02.479 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:02.476+0000 7f4cd72d88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:02.480 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:02.476+0000 7f4cd72d88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:02.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T11:31:02.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:31:02.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:31:02.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:31:02.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:31:02.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:02.606 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:31:02.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:31:02.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:02.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:31:02.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:03.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:03.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:03.756 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:31:03.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:31:03.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:03.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:31:03.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:03.951 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:03.948+0000 7f4cd72d88c0 -1 Falling back to public interface 2026-02-17T11:31:04.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:04.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:04.907 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:31:04.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:31:04.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:04.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:31:05.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:06.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:06.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:06.054 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:31:06.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:31:06.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:06.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:31:06.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:06.905 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:06.900+0000 7f4cd72d88c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:31:07.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:07.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:07.205 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:31:07.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:31:07.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:07.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:31:07.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:08.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:08.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:08.361 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:31:08.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:31:08.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:08.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:31:08.510 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2593518156,v1:127.0.0.1:6811/2593518156] [v2:127.0.0.1:6812/2593518156,v1:127.0.0.1:6813/2593518156] exists,up 622dcc91-6b17-44f5-b56b-782afbf51e4a 2026-02-17T11:31:08.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:31:08.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:31:08.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:31:08.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:548: TEST_auto_repair_bluestore_failed: for id in $(seq 0 2) 2026-02-17T11:31:08.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:549: TEST_auto_repair_bluestore_failed: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:31:08.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:31:08.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:31:08.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T11:31:08.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:31:08.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:31:08.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:31:08.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:31:08.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:31:08.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:31:08.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:31:08.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:31:08.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:31:08.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:31:08.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:31:08.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:31:08.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:31:08.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:31:08.516 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:31:08.516 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:31:08.516 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:31:08.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:31:08.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:31:08.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:31:08.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:31:08.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:31:08.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:31:08.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:31:08.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:31:08.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:31:08.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:31:08.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:31:08.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:31:08.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:31:08.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:31:08.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:31:08.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=1c143ba1-e522-4842-9215-67bc6ca3bbd3 2026-02-17T11:31:08.518 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 1c143ba1-e522-4842-9215-67bc6ca3bbd3 2026-02-17T11:31:08.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 1c143ba1-e522-4842-9215-67bc6ca3bbd3' 2026-02-17T11:31:08.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:31:08.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB8UZRpjcljHxAAPQxgEI4mH/j0WoMz2x7XNA== 2026-02-17T11:31:08.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB8UZRpjcljHxAAPQxgEI4mH/j0WoMz2x7XNA=="}' 2026-02-17T11:31:08.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 1c143ba1-e522-4842-9215-67bc6ca3bbd3 -i td/osd-scrub-repair/2/new.json 2026-02-17T11:31:08.679 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:31:08.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T11:31:08.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQB8UZRpjcljHxAAPQxgEI4mH/j0WoMz2x7XNA== --osd-uuid 1c143ba1-e522-4842-9215-67bc6ca3bbd3 2026-02-17T11:31:08.704 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:08.700+0000 7fd83710f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:08.706 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:08.704+0000 7fd83710f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:08.707 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:08.704+0000 7fd83710f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:08.707 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:08.704+0000 7fd83710f8c0 -1 bdev(0x55fe576b3c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T11:31:08.707 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:08.704+0000 7fd83710f8c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T11:31:11.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T11:31:11.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:31:11.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T11:31:11.252 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T11:31:11.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:31:11.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T11:31:11.442 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T11:31:11.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:31:11.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:31:11.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:31:11.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:31:11.467 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:11.464+0000 7f20b93658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:11.468 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:11.464+0000 7f20b93658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:11.470 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:11.468+0000 7f20b93658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:11.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T11:31:11.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:31:11.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:31:11.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:31:11.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:31:11.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:11.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:31:11.598 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:31:11.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:11.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:31:11.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:12.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:12.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:12.749 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:31:12.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:31:12.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:12.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:31:12.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:12.963 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:12.960+0000 7f20b93658c0 -1 Falling back to public interface 2026-02-17T11:31:13.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:13.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:13.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:31:13.899 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:31:13.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:13.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:31:14.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:15.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:15.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:15.049 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:31:15.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:31:15.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:15.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:31:15.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:15.948 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:15.944+0000 7f20b93658c0 -1 osd.2 0 log_to_monitors true 2026-02-17T11:31:16.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:16.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:16.196 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:31:16.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:31:16.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:16.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:31:16.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:17.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:17.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:17.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:31:17.353 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:31:17.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:17.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:31:17.503 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2273543747,v1:127.0.0.1:6819/2273543747] [v2:127.0.0.1:6820/2273543747,v1:127.0.0.1:6821/2273543747] exists,up 1c143ba1-e522-4842-9215-67bc6ca3bbd3 2026-02-17T11:31:17.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:31:17.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:31:17.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:31:17.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:552: TEST_auto_repair_bluestore_failed: create_pool testpool 1 1 2026-02-17T11:31:17.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create testpool 1 1 2026-02-17T11:31:17.700 INFO:tasks.workunit.client.0.trial128.stderr:pool 'testpool' created 2026-02-17T11:31:17.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:31:18.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:553: TEST_auto_repair_bluestore_failed: ceph osd pool set testpool size 2 2026-02-17T11:31:18.902 INFO:tasks.workunit.client.0.trial128.stderr:set pool 1 size to 2 2026-02-17T11:31:18.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:554: TEST_auto_repair_bluestore_failed: wait_for_clean 2026-02-17T11:31:18.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:31:18.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:31:18.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:31:18.913 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:31:18.914 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:31:18.914 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:31:18.914 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:31:18.914 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:31:18.914 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:31:19.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:31:19.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:31:19.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:31:19.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:31:19.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:31:19.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:31:19.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:31:19.175 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:31:19.175 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:31:19.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:31:19.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:19.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:31:19.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T11:31:19.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T11:31:19.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T11:31:19.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:19.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:31:19.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705668 2026-02-17T11:31:19.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705668 2026-02-17T11:31:19.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705668' 2026-02-17T11:31:19.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:19.321 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:31:19.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542146 2026-02-17T11:31:19.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542146 2026-02-17T11:31:19.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705668 2-60129542146' 2026-02-17T11:31:19.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:31:19.393 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T11:31:19.393 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:31:19.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:31:19.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T11:31:19.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:31:19.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T11:31:19.396 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836485 2026-02-17T11:31:19.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T11:31:19.397 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:31:19.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T11:31:19.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:31:20.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:31:20.555 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:31:20.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T11:31:20.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:31:20.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705668 2026-02-17T11:31:20.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:31:20.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:31:20.715 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705668 2026-02-17T11:31:20.715 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:31:20.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705668 2026-02-17T11:31:20.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-17T11:31:20.717 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705668 2026-02-17T11:31:20.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:31:20.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-17T11:31:20.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:31:20.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542146 2026-02-17T11:31:20.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:31:20.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:31:20.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542146 2026-02-17T11:31:20.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:31:20.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542146 2026-02-17T11:31:20.874 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542146 2026-02-17T11:31:20.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-17T11:31:20.875 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:31:21.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542146 -lt 60129542146 2026-02-17T11:31:21.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:31:21.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:31:21.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:31:21.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:31:21.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:31:21.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:31:21.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:31:21.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:31:21.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:31:21.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:31:21.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:31:21.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:31:21.380 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:31:21.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:31:21.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:31:21.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:31:21.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:31:21.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:31:21.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:557: TEST_auto_repair_bluestore_failed: local payload=ABCDEF 2026-02-17T11:31:21.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:558: TEST_auto_repair_bluestore_failed: echo ABCDEF 2026-02-17T11:31:21.570 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: seq 1 10 2026-02-17T11:31:21.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T11:31:21.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj1 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:31:21.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T11:31:21.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj2 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:31:21.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T11:31:21.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj3 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:31:21.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T11:31:21.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj4 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:31:21.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T11:31:21.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj5 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:31:21.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T11:31:21.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj6 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:31:21.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T11:31:21.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj7 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:31:21.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T11:31:21.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj8 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:31:21.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T11:31:21.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj9 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:31:21.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:559: TEST_auto_repair_bluestore_failed: for i in $(seq 1 10) 2026-02-17T11:31:21.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:561: TEST_auto_repair_bluestore_failed: rados --pool testpool put obj10 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:31:21.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:566: TEST_auto_repair_bluestore_failed: get_not_primary testpool SOMETHING 2026-02-17T11:31:21.798 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T11:31:21.798 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:31:21.798 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T11:31:21.799 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:31:21.799 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:31:21.799 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:31:21.799 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:31:21.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T11:31:21.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:31:21.954 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T11:31:22.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:566: TEST_auto_repair_bluestore_failed: objectstore_tool td/osd-scrub-repair 0 obj1 remove 2026-02-17T11:31:22.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:31:22.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:31:22.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:31:22.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:31:22.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 obj1 remove 2026-02-17T11:31:22.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:31:22.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:31:22.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:31:22.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:31:22.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:31:22.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:31:22.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:31:22.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:31:22.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:31:22.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:31:23.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:31:23.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 obj1 remove 2026-02-17T11:31:23.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:31:23.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:31:23.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:31:23.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:31:23.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:31:23.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj1 remove 2026-02-17T11:31:24.064 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-17T11:31:24.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:31:24.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:31:24.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:31:24.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:31:24.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:31:24.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:31:24.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:31:24.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:31:24.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:31:24.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:31:24.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:31:24.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:31:24.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:31:24.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:31:24.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:31:24.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:31:24.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:31:24.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:31:24.609 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:31:24.609 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:31:24.609 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:31:24.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:31:24.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:31:24.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:31:24.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:31:24.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:31:24.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:31:24.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:31:24.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:31:24.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:31:24.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:31:24.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:31:24.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:31:24.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:31:24.611 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:31:24.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:31:24.611 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:31:24.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:31:24.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:31:24.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:31:24.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:31:24.628 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:24.623+0000 7f5b016798c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:24.629 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:24.623+0000 7f5b016798c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:24.630 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:24.627+0000 7f5b016798c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:24.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:31:24.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:31:24.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:31:24.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:31:24.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:31:24.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:24.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:31:24.762 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:31:24.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:24.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:31:24.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:25.904 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:25.899+0000 7f5b016798c0 -1 Falling back to public interface 2026-02-17T11:31:25.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:25.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:25.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:31:25.915 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:31:25.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:25.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:31:26.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:27.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:27.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:27.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:31:27.065 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:31:27.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:27.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:31:27.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:28.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:28.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:28.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:31:28.216 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:31:28.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:28.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:31:28.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:28.917 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:28.911+0000 7f5b016798c0 -1 osd.0 19 log_to_monitors true 2026-02-17T11:31:29.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:29.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:29.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:31:29.366 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:31:29.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:29.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:31:29.515 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 23 up_thru 0 down_at 20 last_clean_interval [5,19) [v2:127.0.0.1:6802/2554410230,v1:127.0.0.1:6803/2554410230] [v2:127.0.0.1:6804/2554410230,v1:127.0.0.1:6805/2554410230] exists,up efe9f3a2-7c61-4d13-8795-ddce616dc10e 2026-02-17T11:31:29.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:31:29.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:31:29.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:31:29.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:31:29.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:31:29.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:31:29.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:31:29.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:31:29.516 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:31:29.516 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:31:29.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:31:29.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:31:29.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:31:29.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:31:29.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:31:29.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:31:29.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:31:29.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:31:29.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:31:29.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:31:29.783 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:31:29.783 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:31:29.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:31:29.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:29.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:31:29.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247810 2026-02-17T11:31:29.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247810 2026-02-17T11:31:29.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810' 2026-02-17T11:31:29.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:29.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:31:29.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705671 2026-02-17T11:31:29.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705671 2026-02-17T11:31:29.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810 1-38654705671' 2026-02-17T11:31:29.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:29.929 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:31:30.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542149 2026-02-17T11:31:30.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542149 2026-02-17T11:31:30.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810 1-38654705671 2-60129542149' 2026-02-17T11:31:30.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:31:30.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247810 2026-02-17T11:31:30.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:31:30.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:31:30.003 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247810 2026-02-17T11:31:30.003 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:31:30.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247810 2026-02-17T11:31:30.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247810' 2026-02-17T11:31:30.005 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 98784247810 2026-02-17T11:31:30.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:31:30.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 98784247810 2026-02-17T11:31:30.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:31:31.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:31:31.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:31:31.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247810 -lt 98784247810 2026-02-17T11:31:31.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:31:31.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705671 2026-02-17T11:31:31.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:31:31.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:31:31.311 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705671 2026-02-17T11:31:31.311 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:31:31.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705671 2026-02-17T11:31:31.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-17T11:31:31.312 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705671 2026-02-17T11:31:31.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:31:31.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705671 -lt 38654705671 2026-02-17T11:31:31.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:31:31.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542149 2026-02-17T11:31:31.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:31:31.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:31:31.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542149 2026-02-17T11:31:31.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:31:31.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542149 2026-02-17T11:31:31.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542149' 2026-02-17T11:31:31.477 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542149 2026-02-17T11:31:31.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:31:31.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542149 -lt 60129542149 2026-02-17T11:31:31.640 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:31:31.641 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:31:31.641 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:31:31.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:31:31.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:31:31.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:31:31.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:31:31.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:31:31.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:31:31.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:31:31.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:31:31.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:31:31.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:31:31.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:31:31.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:31:32.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:31:32.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:31:32.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:31:32.151 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:568: TEST_auto_repair_bluestore_failed: get_not_primary testpool SOMETHING 2026-02-17T11:31:32.151 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T11:31:32.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:31:32.152 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T11:31:32.153 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:31:32.153 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:31:32.153 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:31:32.153 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:31:32.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T11:31:32.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:31:32.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T11:31:32.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:568: TEST_auto_repair_bluestore_failed: objectstore_tool td/osd-scrub-repair 0 obj2 remove 2026-02-17T11:31:32.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:31:32.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:31:32.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:31:32.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:31:32.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 obj2 remove 2026-02-17T11:31:32.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:31:32.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:31:32.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:31:32.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:31:32.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:31:32.463 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:31:32.463 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:31:32.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:31:32.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:31:32.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:31:33.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:31:33.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 obj2 remove 2026-02-17T11:31:33.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:31:33.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:31:33.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:31:33.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:31:33.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:31:33.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj2 remove 2026-02-17T11:31:34.403 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:104778fc:::obj2:head# 2026-02-17T11:31:34.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:31:34.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:31:34.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:31:34.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:31:34.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:31:34.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:31:34.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:31:34.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:31:34.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:31:34.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:31:34.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:31:34.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:31:34.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:31:34.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:31:34.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:31:34.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:31:34.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:31:34.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:31:34.952 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:31:34.952 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:31:34.952 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:31:34.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:31:34.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:31:34.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:31:34.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:31:34.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:31:34.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:31:34.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:31:34.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:31:34.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:31:34.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:31:34.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:31:34.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:31:34.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:31:34.955 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:31:34.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:31:34.956 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:31:34.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:31:34.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:31:34.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:31:34.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:31:34.970 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:34.967+0000 7f2c118f28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:34.972 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:34.967+0000 7f2c118f28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:34.972 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:34.967+0000 7f2c118f28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:35.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:31:35.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:31:35.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:31:35.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:31:35.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:31:35.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:35.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:31:35.104 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:31:35.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:35.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:31:35.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:36.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:36.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:36.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:31:36.262 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:31:36.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:36.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:31:36.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:36.443 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:36.439+0000 7f2c118f28c0 -1 Falling back to public interface 2026-02-17T11:31:37.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:37.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:37.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:31:37.417 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:31:37.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:37.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:31:37.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:38.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:38.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:38.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:31:38.572 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:31:38.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:38.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:31:38.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:39.446 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:39.443+0000 7f2c118f28c0 -1 osd.0 24 log_to_monitors true 2026-02-17T11:31:39.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:39.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:39.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:31:39.729 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:31:39.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:39.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:31:39.882 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 29 up_thru 0 down_at 26 last_clean_interval [23,24) [v2:127.0.0.1:6802/1185863916,v1:127.0.0.1:6803/1185863916] [v2:127.0.0.1:6804/1185863916,v1:127.0.0.1:6805/1185863916] exists,up efe9f3a2-7c61-4d13-8795-ddce616dc10e 2026-02-17T11:31:39.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:31:39.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:31:39.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:31:39.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:31:39.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:31:39.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:31:39.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:31:39.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:31:39.884 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:31:39.884 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:31:39.884 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:31:39.884 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:31:39.884 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:31:39.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:31:39.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:31:39.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:31:39.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:31:39.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:31:39.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:31:40.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:31:40.144 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:31:40.145 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:31:40.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:31:40.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:40.145 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:31:40.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051586 2026-02-17T11:31:40.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051586 2026-02-17T11:31:40.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586' 2026-02-17T11:31:40.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:40.216 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:31:40.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705674 2026-02-17T11:31:40.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705674 2026-02-17T11:31:40.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586 1-38654705674' 2026-02-17T11:31:40.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:40.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:31:40.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542152 2026-02-17T11:31:40.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542152 2026-02-17T11:31:40.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586 1-38654705674 2-60129542152' 2026-02-17T11:31:40.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:31:40.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-124554051586 2026-02-17T11:31:40.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:31:40.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:31:40.362 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-124554051586 2026-02-17T11:31:40.362 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:31:40.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051586 2026-02-17T11:31:40.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 124554051586' 2026-02-17T11:31:40.363 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 124554051586 2026-02-17T11:31:40.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:31:40.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247810 -lt 124554051586 2026-02-17T11:31:40.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:31:41.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:31:41.519 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:31:41.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051586 -lt 124554051586 2026-02-17T11:31:41.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:31:41.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705674 2026-02-17T11:31:41.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:31:41.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:31:41.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705674 2026-02-17T11:31:41.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:31:41.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705674 2026-02-17T11:31:41.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-17T11:31:41.680 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705674 2026-02-17T11:31:41.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:31:41.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705674 -lt 38654705674 2026-02-17T11:31:41.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:31:41.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542152 2026-02-17T11:31:41.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:31:41.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:31:41.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542152 2026-02-17T11:31:41.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:31:41.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542152 2026-02-17T11:31:41.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542152' 2026-02-17T11:31:41.848 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542152 2026-02-17T11:31:41.849 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:31:42.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542152 -lt 60129542152 2026-02-17T11:31:42.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:31:42.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:31:42.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:31:42.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:31:42.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:31:42.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:31:42.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:31:42.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:31:42.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:31:42.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:31:42.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:31:42.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:31:42.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:31:42.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:31:42.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:31:42.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:31:42.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:31:42.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:31:42.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:569: TEST_auto_repair_bluestore_failed: get_primary testpool SOMETHING 2026-02-17T11:31:42.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:31:42.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:31:42.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:31:42.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:31:42.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:569: TEST_auto_repair_bluestore_failed: objectstore_tool td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-17T11:31:42.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:31:42.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:31:42.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:31:42.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:31:42.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-17T11:31:42.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:31:42.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:31:42.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:31:42.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:31:42.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:31:42.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:31:42.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:31:42.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:31:42.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:31:42.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:31:43.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:31:43.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-17T11:31:43.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:31:43.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:31:43.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:31:43.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:31:43.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:31:43.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj2 rm-attr _ 2026-02-17T11:31:45.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:31:45.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:31:45.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:31:45.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:31:45.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:31:45.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:31:45.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:31:45.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:31:45.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:31:45.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:31:45.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:31:45.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:31:45.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:31:45.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:31:45.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:31:45.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:31:45.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:31:45.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:31:45.171 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:31:45.171 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:31:45.171 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:31:45.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:31:45.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:31:45.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:31:45.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:31:45.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:31:45.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:31:45.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:31:45.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:31:45.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:31:45.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:31:45.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:31:45.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:31:45.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:31:45.175 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:31:45.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:31:45.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:31:45.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:31:45.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:31:45.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:31:45.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:31:45.195 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:45.191+0000 7f455c0aa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:45.196 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:45.191+0000 7f455c0aa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:45.197 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:45.191+0000 7f455c0aa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:31:45.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:31:45.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:31:45.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:31:45.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:31:45.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:31:45.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:45.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:31:45.328 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:31:45.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:45.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:31:45.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:46.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:46.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:46.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:31:46.485 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:31:46.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:46.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:31:46.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:46.656 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:46.651+0000 7f455c0aa8c0 -1 Falling back to public interface 2026-02-17T11:31:47.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:47.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:47.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:31:47.639 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:31:47.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:47.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:31:47.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:48.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:48.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:48.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:31:48.792 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:31:48.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:48.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:31:48.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:31:49.610 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:31:49.603+0000 7f455c0aa8c0 -1 osd.1 30 log_to_monitors true 2026-02-17T11:31:49.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:31:49.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:31:49.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:31:49.942 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:31:49.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:31:49.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:31:50.092 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 34 up_thru 34 down_at 31 last_clean_interval [9,30) [v2:127.0.0.1:6810/1042927659,v1:127.0.0.1:6811/1042927659] [v2:127.0.0.1:6812/1042927659,v1:127.0.0.1:6813/1042927659] exists,up 622dcc91-6b17-44f5-b56b-782afbf51e4a 2026-02-17T11:31:50.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:31:50.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:31:50.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:31:50.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:31:50.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:31:50.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:31:50.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:31:50.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:31:50.094 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:31:50.094 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:31:50.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:31:50.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:31:50.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:31:50.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:31:50.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:31:50.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:31:50.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:31:50.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:31:50.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:31:50.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:31:50.340 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:31:50.340 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:31:50.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:31:50.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:50.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:31:50.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051589 2026-02-17T11:31:50.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051589 2026-02-17T11:31:50.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051589' 2026-02-17T11:31:50.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:50.411 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:31:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888066 2026-02-17T11:31:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888066 2026-02-17T11:31:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051589 1-146028888066' 2026-02-17T11:31:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:50.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:31:50.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542155 2026-02-17T11:31:50.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542155 2026-02-17T11:31:50.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051589 1-146028888066 2-60129542155' 2026-02-17T11:31:50.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:31:50.554 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-124554051589 2026-02-17T11:31:50.554 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:31:50.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:31:50.555 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-124554051589 2026-02-17T11:31:50.555 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:31:50.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051589 2026-02-17T11:31:50.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 124554051589' 2026-02-17T11:31:50.557 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 124554051589 2026-02-17T11:31:50.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:31:50.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051589 -lt 124554051589 2026-02-17T11:31:50.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:31:50.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-146028888066 2026-02-17T11:31:50.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:31:50.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:31:50.715 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-146028888066 2026-02-17T11:31:50.715 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:31:50.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888066 2026-02-17T11:31:50.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 146028888066' 2026-02-17T11:31:50.717 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 146028888066 2026-02-17T11:31:50.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:31:50.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888066 -lt 146028888066 2026-02-17T11:31:50.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:31:50.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542155 2026-02-17T11:31:50.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:31:50.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:31:50.880 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542155 2026-02-17T11:31:50.880 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:31:50.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542155 2026-02-17T11:31:50.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542155' 2026-02-17T11:31:50.882 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542155 2026-02-17T11:31:50.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:31:51.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542155 -lt 60129542155 2026-02-17T11:31:51.049 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:31:51.049 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:31:51.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:31:51.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:31:51.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:31:51.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:31:51.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:31:51.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:31:51.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:31:51.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:31:51.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:31:51.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:31:51.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:31:51.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:31:51.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:31:51.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:31:51.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:31:51.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:31:51.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:571: TEST_auto_repair_bluestore_failed: get_pg testpool obj1 2026-02-17T11:31:51.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=testpool 2026-02-17T11:31:51.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=obj1 2026-02-17T11:31:51.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map testpool obj1 2026-02-17T11:31:51.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:31:51.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:571: TEST_auto_repair_bluestore_failed: local pgid=1.0 2026-02-17T11:31:51.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:572: TEST_auto_repair_bluestore_failed: get_primary testpool obj1 2026-02-17T11:31:51.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:31:51.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T11:31:51.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool obj1 2026-02-17T11:31:51.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:31:51.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:572: TEST_auto_repair_bluestore_failed: local primary=1 2026-02-17T11:31:51.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:573: TEST_auto_repair_bluestore_failed: get_last_scrub_stamp 1.0 2026-02-17T11:31:51.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:31:51.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:31:51.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:31:51.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:31:52.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:573: TEST_auto_repair_bluestore_failed: local last_scrub_stamp=2026-02-17T11:31:17.698594+0000 2026-02-17T11:31:52.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:574: TEST_auto_repair_bluestore_failed: ceph tell 1.0 schedule-deep-scrub 2026-02-17T11:31:52.107 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T11:31:52.108 INFO:tasks.workunit.client.0.trial128.stdout: "deep": true, 2026-02-17T11:31:52.108 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T11:31:52.108 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-03T11:30:12.106970+0000" 2026-02-17T11:31:52.108 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:31:52.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:577: TEST_auto_repair_bluestore_failed: wait_for_scrub 1.0 2026-02-17T11:31:17.698594+0000 2026-02-17T11:31:52.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T11:31:52.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:31:17.698594+0000 2026-02-17T11:31:52.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:31:52.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:31:52.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:31:52.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:31:52.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:31:52.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:31:52.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:31:52.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:31:52.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:31:17.698594+0000 '>' 2026-02-17T11:31:17.698594+0000 2026-02-17T11:31:52.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:31:53.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:31:53.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:31:53.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:31:53.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:31:53.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:31:53.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:31:53.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:31:53.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:31:17.698594+0000 '>' 2026-02-17T11:31:17.698594+0000 2026-02-17T11:31:53.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:31:54.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:31:54.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:31:54.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:31:54.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:31:54.429 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:31:54.429 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:31:54.429 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:31:54.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:31:17.698594+0000 '>' 2026-02-17T11:31:17.698594+0000 2026-02-17T11:31:54.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:31:55.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:31:55.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:31:55.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:31:55.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:31:55.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:31:55.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:31:55.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:31:55.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:31:17.698594+0000 '>' 2026-02-17T11:31:17.698594+0000 2026-02-17T11:31:55.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:31:56.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:31:56.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:31:56.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:31:56.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:31:56.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:31:56.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:31:56.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:31:56.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:31:52.634370+0000 '>' 2026-02-17T11:31:17.698594+0000 2026-02-17T11:31:56.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:31:56.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:578: TEST_auto_repair_bluestore_failed: wait_for_clean 2026-02-17T11:31:56.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:31:56.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:31:56.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:31:56.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:31:56.883 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:31:56.883 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:31:56.884 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:31:56.884 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:31:56.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:31:56.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:31:56.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:31:56.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:31:56.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:31:56.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:31:56.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:31:57.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:31:57.131 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:31:57.131 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:31:57.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:31:57.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:57.132 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:31:57.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051591 2026-02-17T11:31:57.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051591 2026-02-17T11:31:57.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051591' 2026-02-17T11:31:57.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:57.205 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:31:57.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888068 2026-02-17T11:31:57.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888068 2026-02-17T11:31:57.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051591 1-146028888068' 2026-02-17T11:31:57.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:57.276 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:31:57.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542158 2026-02-17T11:31:57.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542158 2026-02-17T11:31:57.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051591 1-146028888068 2-60129542158' 2026-02-17T11:31:57.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:31:57.350 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-124554051591 2026-02-17T11:31:57.350 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:31:57.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:31:57.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-124554051591 2026-02-17T11:31:57.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:31:57.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051591 2026-02-17T11:31:57.354 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 124554051591 2026-02-17T11:31:57.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 124554051591' 2026-02-17T11:31:57.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:31:57.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051590 -lt 124554051591 2026-02-17T11:31:57.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:31:58.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:31:58.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:31:58.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051591 -lt 124554051591 2026-02-17T11:31:58.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:31:58.672 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-146028888068 2026-02-17T11:31:58.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:31:58.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:31:58.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-146028888068 2026-02-17T11:31:58.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:31:58.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888068 2026-02-17T11:31:58.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 146028888068' 2026-02-17T11:31:58.677 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 146028888068 2026-02-17T11:31:58.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:31:58.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888068 -lt 146028888068 2026-02-17T11:31:58.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:31:58.840 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542158 2026-02-17T11:31:58.840 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:31:58.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:31:58.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542158 2026-02-17T11:31:58.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:31:58.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542158 2026-02-17T11:31:58.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542158' 2026-02-17T11:31:58.844 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542158 2026-02-17T11:31:58.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:31:58.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542158 -lt 60129542158 2026-02-17T11:31:58.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:31:58.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:31:58.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:31:59.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:31:59.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:31:59.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:31:59.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:31:59.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:31:59.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:31:59.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:31:59.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:31:59.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:31:59.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:31:59.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:31:59.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:31:59.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:31:59.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:31:59.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:31:59.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:579: TEST_auto_repair_bluestore_failed: flush_pg_stats 2026-02-17T11:31:59.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:31:59.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:31:59.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:31:59.676 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:31:59.676 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:31:59.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:31:59.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:59.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:31:59.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051593 2026-02-17T11:31:59.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051593 2026-02-17T11:31:59.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051593' 2026-02-17T11:31:59.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:59.732 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:31:59.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888070 2026-02-17T11:31:59.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888070 2026-02-17T11:31:59.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051593 1-146028888070' 2026-02-17T11:31:59.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:31:59.804 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:31:59.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542159 2026-02-17T11:31:59.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542159 2026-02-17T11:31:59.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051593 1-146028888070 2-60129542159' 2026-02-17T11:31:59.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:31:59.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-124554051593 2026-02-17T11:31:59.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:31:59.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:31:59.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-124554051593 2026-02-17T11:31:59.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:31:59.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051593 2026-02-17T11:31:59.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 124554051593' 2026-02-17T11:31:59.876 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 124554051593 2026-02-17T11:31:59.876 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:32:00.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051591 -lt 124554051593 2026-02-17T11:32:00.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:32:01.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:32:01.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:32:01.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051593 -lt 124554051593 2026-02-17T11:32:01.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:32:01.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-146028888070 2026-02-17T11:32:01.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:32:01.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:32:01.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-146028888070 2026-02-17T11:32:01.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:32:01.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888070 2026-02-17T11:32:01.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 146028888070' 2026-02-17T11:32:01.194 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 146028888070 2026-02-17T11:32:01.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:32:01.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888070 -lt 146028888070 2026-02-17T11:32:01.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:32:01.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542159 2026-02-17T11:32:01.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:32:01.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:32:01.361 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542159 2026-02-17T11:32:01.362 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:32:01.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542159 2026-02-17T11:32:01.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542159' 2026-02-17T11:32:01.363 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542159 2026-02-17T11:32:01.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:32:01.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542159 -lt 60129542159 2026-02-17T11:32:01.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:580: TEST_auto_repair_bluestore_failed: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-17T11:32:01.517 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.623+0000 7f453a028640 10 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] planned AUTO_REPAIR TIME_FOR_DEEP] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T11:32:01.517 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.623+0000 7f453a028640 10 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-17T11:32:01.517 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.627+0000 7f453a028640 15 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish: 5 errors. 2 errors fixed 2026-02-17T11:32:01.517 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.627+0000 7f453a028640 20 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish Current 'required': 0 Planned 'req_scrub': 0 2026-02-17T11:32:01.517 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.627+0000 7f453a028640 19 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T11:32:01.517 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.631+0000 7f453e030640 10 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] planned CHECK_REPAIR MUST_DEEP_SCRUB MUST_SCRUB] scrubber: scrub_finish before flags: CHECK_REPAIR REQ_SCRUB. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T11:32:01.517 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.631+0000 7f453e030640 10 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 0 2026-02-17T11:32:01.517 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.631+0000 7f453e030640 19 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] ] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T11:32:01.517 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.631+0000 7f453e030640 10 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent+failed_repair [ 1.0: REQ_SCRUB ] ] scrubber: scrub_finish 3 error(s) still present after re-scrub 2026-02-17T11:32:01.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:581: TEST_auto_repair_bluestore_failed: grep -q 'scrub_finish.*still present after re-scrub' td/osd-scrub-repair/osd.1.log 2026-02-17T11:32:01.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:582: TEST_auto_repair_bluestore_failed: ceph pg dump pgs 2026-02-17T11:32:01.671 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T11:32:01.671 INFO:tasks.workunit.client.0.trial128.stdout:1.0 10 0 0 0 0 63 0 0 10 0 10 active+clean+inconsistent+failed_repair 2026-02-17T11:31:52.634427+0000 19'10 35:102 [1,0] 1 [1,0] 1 19'10 2026-02-17T11:31:52.634370+0000 19'10 2026-02-17T11:31:52.634370+0000 0 1 periodic scrub scheduled @ 2026-02-18T11:31:52.634370+0000 10 0 2026-02-17T11:32:01.671 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:32:01.672 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T11:32:01.672 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:32:01.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:583: TEST_auto_repair_bluestore_failed: ceph pg dump pgs 2026-02-17T11:32:01.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:583: TEST_auto_repair_bluestore_failed: grep -q '^1.0.*+failed_repair' 2026-02-17T11:32:01.836 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:32:01.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:587: TEST_auto_repair_bluestore_failed: get_not_primary testpool obj1 2026-02-17T11:32:01.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T11:32:01.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=obj1 2026-02-17T11:32:01.845 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool obj1 2026-02-17T11:32:01.845 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:32:01.845 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T11:32:01.846 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool obj1 2026-02-17T11:32:01.846 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:32:01.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T11:32:01.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool obj1 2026-02-17T11:32:01.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T11:32:02.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:587: TEST_auto_repair_bluestore_failed: objectstore_tool td/osd-scrub-repair 0 obj1 list-attrs 2026-02-17T11:32:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:32:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:32:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:32:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:32:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 obj1 list-attrs 2026-02-17T11:32:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:32:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:32:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:32:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:32:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:32:02.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:32:02.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:32:02.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:32:02.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:32:02.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:32:03.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:32:03.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 obj1 list-attrs 2026-02-17T11:32:03.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:32:03.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:32:03.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:32:03.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:32:03.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:32:03.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj1 list-attrs 2026-02-17T11:32:03.790 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:32:03.790 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:32:04.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:32:04.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:32:04.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:32:04.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:32:04.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:32:04.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:32:04.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:32:04.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:32:04.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:32:04.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:32:04.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:32:04.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:32:04.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:32:04.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:32:04.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:32:04.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:32:04.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:32:04.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:32:04.324 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:32:04.324 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:32:04.324 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:32:04.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:32:04.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:32:04.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:32:04.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:32:04.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:32:04.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:32:04.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:32:04.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:32:04.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:32:04.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:32:04.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:32:04.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:32:04.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:32:04.326 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:32:04.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:32:04.326 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:32:04.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:32:04.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:32:04.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:32:04.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:32:04.344 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:04.338+0000 7f6f410818c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:04.345 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:04.338+0000 7f6f410818c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:04.346 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:04.342+0000 7f6f410818c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:04.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:32:04.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:32:04.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:32:04.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:32:04.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:32:04.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:04.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:32:04.481 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:32:04.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:04.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:32:04.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:05.592 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:05.586+0000 7f6f410818c0 -1 Falling back to public interface 2026-02-17T11:32:05.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:05.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:05.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:32:05.633 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:32:05.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:05.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:32:05.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:06.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:06.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:06.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:32:06.785 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:32:06.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:06.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:32:06.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:07.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:07.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:07.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:32:07.935 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:32:07.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:07.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:32:08.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:08.607 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:08.602+0000 7f6f410818c0 -1 osd.0 35 log_to_monitors true 2026-02-17T11:32:09.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:09.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:09.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:32:09.087 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:32:09.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:09.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:32:09.238 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 39 up_thru 31 down_at 36 last_clean_interval [29,35) [v2:127.0.0.1:6802/3669783729,v1:127.0.0.1:6803/3669783729] [v2:127.0.0.1:6804/3669783729,v1:127.0.0.1:6805/3669783729] exists,up efe9f3a2-7c61-4d13-8795-ddce616dc10e 2026-02-17T11:32:09.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:32:09.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:32:09.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:32:09.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:32:09.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:32:09.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:32:09.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:32:09.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:32:09.239 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:32:09.239 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:32:09.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:32:09.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:32:09.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:32:09.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:32:09.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:32:09.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:32:09.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:32:09.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:32:09.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:32:09.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:32:09.500 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:32:09.500 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:32:09.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:32:09.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:32:09.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:32:09.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724546 2026-02-17T11:32:09.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724546 2026-02-17T11:32:09.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724546' 2026-02-17T11:32:09.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:32:09.572 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:32:09.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888072 2026-02-17T11:32:09.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888072 2026-02-17T11:32:09.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724546 1-146028888072' 2026-02-17T11:32:09.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:32:09.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:32:09.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542162 2026-02-17T11:32:09.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542162 2026-02-17T11:32:09.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724546 1-146028888072 2-60129542162' 2026-02-17T11:32:09.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:32:09.716 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-167503724546 2026-02-17T11:32:09.716 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:32:09.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:32:09.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-167503724546 2026-02-17T11:32:09.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:32:09.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724546 2026-02-17T11:32:09.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 167503724546' 2026-02-17T11:32:09.720 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 167503724546 2026-02-17T11:32:09.720 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:32:09.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051593 -lt 167503724546 2026-02-17T11:32:09.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:32:10.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:32:10.877 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:32:11.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724546 -lt 167503724546 2026-02-17T11:32:11.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:32:11.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-146028888072 2026-02-17T11:32:11.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:32:11.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:32:11.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-146028888072 2026-02-17T11:32:11.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:32:11.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888072 2026-02-17T11:32:11.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 146028888072' 2026-02-17T11:32:11.037 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 146028888072 2026-02-17T11:32:11.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:32:11.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888073 -lt 146028888072 2026-02-17T11:32:11.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:32:11.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542162 2026-02-17T11:32:11.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:32:11.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:32:11.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542162 2026-02-17T11:32:11.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:32:11.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542162 2026-02-17T11:32:11.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542162' 2026-02-17T11:32:11.197 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542162 2026-02-17T11:32:11.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:32:11.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542162 -lt 60129542162 2026-02-17T11:32:11.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:32:11.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:32:11.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:32:11.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:32:11.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:32:11.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:32:11.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:32:11.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:32:11.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:32:11.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:32:11.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:32:11.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:32:11.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:32:11.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:32:11.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:32:11.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:32:11.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:32:11.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:32:11.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:588: TEST_auto_repair_bluestore_failed: rados --pool testpool get obj1 td/osd-scrub-repair/COPY 2026-02-17T11:32:11.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:589: TEST_auto_repair_bluestore_failed: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:32:11.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:590: TEST_auto_repair_bluestore_failed: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-17T11:32:11.914 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.623+0000 7f453a028640 10 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] planned AUTO_REPAIR TIME_FOR_DEEP] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T11:32:11.914 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.623+0000 7f453a028640 10 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-17T11:32:11.914 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.627+0000 7f453a028640 15 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish: 5 errors. 2 errors fixed 2026-02-17T11:32:11.915 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.627+0000 7f453a028640 20 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish Current 'required': 0 Planned 'req_scrub': 0 2026-02-17T11:32:11.915 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.627+0000 7f453a028640 19 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T11:32:11.915 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.631+0000 7f453e030640 10 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] planned CHECK_REPAIR MUST_DEEP_SCRUB MUST_SCRUB] scrubber: scrub_finish before flags: CHECK_REPAIR REQ_SCRUB. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T11:32:11.915 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.631+0000 7f453e030640 10 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 0 2026-02-17T11:32:11.915 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.631+0000 7f453e030640 19 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] ] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T11:32:11.915 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.631+0000 7f453e030640 10 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent+failed_repair [ 1.0: REQ_SCRUB ] ] scrubber: scrub_finish 3 error(s) still present after re-scrub 2026-02-17T11:32:11.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:593: TEST_auto_repair_bluestore_failed: get_primary testpool SOMETHING 2026-02-17T11:32:11.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:32:11.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:32:11.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:32:11.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:32:12.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:593: TEST_auto_repair_bluestore_failed: objectstore_tool td/osd-scrub-repair 1 obj2 remove 2026-02-17T11:32:12.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:32:12.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:32:12.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:32:12.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:32:12.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 obj2 remove 2026-02-17T11:32:12.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:32:12.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:32:12.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:32:12.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:32:12.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:32:12.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:32:12.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:32:12.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:32:12.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:32:12.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:32:13.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:32:13.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 obj2 remove 2026-02-17T11:32:13.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:32:13.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:32:13.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:32:13.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:32:13.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:32:13.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj2 remove 2026-02-17T11:32:14.024 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:104778fc:::obj2:head#, (61) No data available 2026-02-17T11:32:14.024 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:104778fc:::obj2:head# 2026-02-17T11:32:14.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:32:14.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:32:14.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:32:14.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:32:14.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:32:14.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:32:14.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:32:14.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:32:14.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:32:14.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:32:14.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:32:14.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:32:14.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:32:14.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:32:14.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:32:14.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:32:14.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:32:14.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:32:14.565 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:32:14.565 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:32:14.565 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:32:14.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:32:14.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:32:14.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:32:14.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:32:14.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:32:14.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:32:14.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:32:14.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:32:14.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:32:14.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:32:14.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:32:14.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:32:14.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:32:14.566 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:32:14.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:32:14.566 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:32:14.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:32:14.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:32:14.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:32:14.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:32:14.588 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:14.582+0000 7fd64511e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:14.589 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:14.582+0000 7fd64511e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:14.590 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:14.582+0000 7fd64511e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:14.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:32:14.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:32:14.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:32:14.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:32:14.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:32:14.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:14.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:32:14.717 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:32:14.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:14.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:32:14.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:15.849 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:15.842+0000 7fd64511e8c0 -1 Falling back to public interface 2026-02-17T11:32:15.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:15.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:15.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:32:15.872 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:32:15.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:15.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:32:16.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:17.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:17.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:17.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:32:17.024 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:32:17.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:17.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:32:17.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:18.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:18.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:18.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:32:18.182 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:32:18.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:18.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:32:18.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:18.837 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:18.830+0000 7fd64511e8c0 -1 osd.1 40 log_to_monitors true 2026-02-17T11:32:19.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:19.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:19.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:32:19.340 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:32:19.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:19.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:32:19.494 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 44 up_thru 44 down_at 41 last_clean_interval [34,40) [v2:127.0.0.1:6810/2378907798,v1:127.0.0.1:6811/2378907798] [v2:127.0.0.1:6812/2378907798,v1:127.0.0.1:6813/2378907798] exists,up 622dcc91-6b17-44f5-b56b-782afbf51e4a 2026-02-17T11:32:19.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:32:19.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:32:19.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:32:19.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:32:19.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:32:19.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:32:19.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:32:19.494 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:32:19.495 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:32:19.495 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:32:19.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:32:19.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:32:19.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:32:19.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:32:19.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:32:19.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:32:19.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:32:19.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:32:19.604 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:32:19.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:32:19.763 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:32:19.763 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:32:19.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:32:19.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:32:19.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:32:19.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724549 2026-02-17T11:32:19.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724549 2026-02-17T11:32:19.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724549' 2026-02-17T11:32:19.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:32:19.836 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:32:19.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561026 2026-02-17T11:32:19.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561026 2026-02-17T11:32:19.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724549 1-188978561026' 2026-02-17T11:32:19.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:32:19.906 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:32:19.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542165 2026-02-17T11:32:19.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542165 2026-02-17T11:32:19.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724549 1-188978561026 2-60129542165' 2026-02-17T11:32:19.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:32:19.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-167503724549 2026-02-17T11:32:19.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:32:19.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:32:19.980 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-167503724549 2026-02-17T11:32:19.980 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:32:19.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724549 2026-02-17T11:32:19.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 167503724549' 2026-02-17T11:32:19.982 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 167503724549 2026-02-17T11:32:19.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:32:20.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724547 -lt 167503724549 2026-02-17T11:32:20.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:32:21.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:32:21.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:32:21.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724549 -lt 167503724549 2026-02-17T11:32:21.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:32:21.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-188978561026 2026-02-17T11:32:21.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:32:21.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:32:21.299 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-188978561026 2026-02-17T11:32:21.299 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:32:21.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561026 2026-02-17T11:32:21.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 188978561026' 2026-02-17T11:32:21.300 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 188978561026 2026-02-17T11:32:21.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:32:21.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561026 -lt 188978561026 2026-02-17T11:32:21.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:32:21.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542165 2026-02-17T11:32:21.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:32:21.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:32:21.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542165 2026-02-17T11:32:21.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:32:21.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542165 2026-02-17T11:32:21.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542165' 2026-02-17T11:32:21.469 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542165 2026-02-17T11:32:21.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:32:21.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542165 -lt 60129542165 2026-02-17T11:32:21.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:32:21.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:32:21.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:32:21.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:32:21.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:32:21.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:32:21.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:32:21.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:32:21.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:32:21.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:32:21.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:32:21.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:32:21.956 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:32:21.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:32:21.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:32:22.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:32:22.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:32:22.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:32:22.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:594: TEST_auto_repair_bluestore_failed: repair 1.0 2026-02-17T11:32:22.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.0 2026-02-17T11:32:22.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.0 2026-02-17T11:32:22.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:32:22.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:32:22.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:32:22.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:32:22.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:31:52.634370+0000 2026-02-17T11:32:22.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.0 2026-02-17T11:32:22.433 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 1.0 on osd.1 to repair 2026-02-17T11:32:22.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.0 2026-02-17T11:31:52.634370+0000 2026-02-17T11:32:22.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T11:32:22.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:31:52.634370+0000 2026-02-17T11:32:22.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:32:22.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:32:22.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:32:22.443 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:32:22.443 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:32:22.443 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:32:22.443 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:32:22.443 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:32:22.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:31:52.634370+0000 '>' 2026-02-17T11:31:52.634370+0000 2026-02-17T11:32:22.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:32:23.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:32:23.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:32:23.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:32:23.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:32:23.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:32:23.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:32:23.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:32:23.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:31:52.634370+0000 '>' 2026-02-17T11:31:52.634370+0000 2026-02-17T11:32:23.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:32:24.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:32:24.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:32:24.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:32:24.759 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:32:24.759 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:32:24.759 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:32:24.759 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:32:24.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:32:22.982395+0000 '>' 2026-02-17T11:31:52.634370+0000 2026-02-17T11:32:24.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:32:24.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:595: TEST_auto_repair_bluestore_failed: sleep 2 2026-02-17T11:32:26.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:597: TEST_auto_repair_bluestore_failed: flush_pg_stats 2026-02-17T11:32:26.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:32:26.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:32:27.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:32:27.063 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:32:27.063 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:32:27.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:32:27.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:32:27.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:32:27.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724551 2026-02-17T11:32:27.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724551 2026-02-17T11:32:27.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724551' 2026-02-17T11:32:27.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:32:27.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:32:27.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561028 2026-02-17T11:32:27.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561028 2026-02-17T11:32:27.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724551 1-188978561028' 2026-02-17T11:32:27.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:32:27.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:32:27.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542168 2026-02-17T11:32:27.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542168 2026-02-17T11:32:27.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724551 1-188978561028 2-60129542168' 2026-02-17T11:32:27.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:32:27.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-167503724551 2026-02-17T11:32:27.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:32:27.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:32:27.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-167503724551 2026-02-17T11:32:27.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:32:27.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724551 2026-02-17T11:32:27.285 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 167503724551 2026-02-17T11:32:27.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 167503724551' 2026-02-17T11:32:27.286 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:32:27.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724550 -lt 167503724551 2026-02-17T11:32:27.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:32:28.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:32:28.445 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:32:28.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724550 -lt 167503724551 2026-02-17T11:32:28.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:32:29.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:32:29.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:32:29.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724551 -lt 167503724551 2026-02-17T11:32:29.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:32:29.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-188978561028 2026-02-17T11:32:29.759 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:32:29.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:32:29.760 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-188978561028 2026-02-17T11:32:29.761 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:32:29.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561028 2026-02-17T11:32:29.762 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 188978561028 2026-02-17T11:32:29.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 188978561028' 2026-02-17T11:32:29.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:32:29.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561028 -lt 188978561028 2026-02-17T11:32:29.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:32:29.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542168 2026-02-17T11:32:29.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:32:29.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:32:29.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542168 2026-02-17T11:32:29.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:32:29.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542168 2026-02-17T11:32:29.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542168' 2026-02-17T11:32:29.925 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542168 2026-02-17T11:32:29.925 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:32:30.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542168 -lt 60129542168 2026-02-17T11:32:30.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:598: TEST_auto_repair_bluestore_failed: ceph pg dump pgs 2026-02-17T11:32:30.227 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T11:32:30.227 INFO:tasks.workunit.client.0.trial128.stdout:1.0 9 0 0 0 0 63 0 0 10 0 10 active+clean 2026-02-17T11:32:22.982492+0000 19'10 45:140 [1,0] 1 [1,0] 1 19'10 2026-02-17T11:32:22.982395+0000 19'10 2026-02-17T11:32:22.982395+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:32:22.982395+0000 9 0 2026-02-17T11:32:30.227 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:32:30.227 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T11:32:30.228 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:32:30.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:599: TEST_auto_repair_bluestore_failed: ceph pg dump pgs 2026-02-17T11:32:30.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:599: TEST_auto_repair_bluestore_failed: grep -q -e '^1.0.* active+clean ' -e '^1.0.* active+clean+wait ' 2026-02-17T11:32:30.387 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:32:30.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:600: TEST_auto_repair_bluestore_failed: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-17T11:32:30.398 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.623+0000 7f453a028640 10 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] planned AUTO_REPAIR TIME_FOR_DEEP] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T11:32:30.398 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.623+0000 7f453a028640 10 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-17T11:32:30.398 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.627+0000 7f453a028640 15 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish: 5 errors. 2 errors fixed 2026-02-17T11:32:30.398 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.627+0000 7f453a028640 20 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish Current 'required': 0 Planned 'req_scrub': 0 2026-02-17T11:32:30.398 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.627+0000 7f453a028640 19 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T11:32:30.398 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.631+0000 7f453e030640 10 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] planned CHECK_REPAIR MUST_DEEP_SCRUB MUST_SCRUB] scrubber: scrub_finish before flags: CHECK_REPAIR REQ_SCRUB. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T11:32:30.398 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.631+0000 7f453e030640 10 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 0 2026-02-17T11:32:30.398 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.631+0000 7f453e030640 19 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: CHECK_REPAIR REQ_SCRUB ] ] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T11:32:30.398 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:31:52.631+0000 7f453e030640 10 osd.1 pg_epoch: 35 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=34/35 n=10 ec=15/15 lis/c=34/34 les/c/f=35/35/0 sis=34) [1,0] r=0 lpr=34 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent+failed_repair [ 1.0: REQ_SCRUB ] ] scrubber: scrub_finish 3 error(s) still present after re-scrub 2026-02-17T11:32:30.398 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:32:22.978+0000 7fd6270a4640 10 osd.1 pg_epoch: 45 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=44/45 n=10 ec=15/15 lis/c=44/44 les/c/f=45/45/0 sis=44) [1,0] r=0 lpr=44 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+repair [ 1.0: AUTO_REPAIR REQ_SCRUB ] MUST_REPAIR planned AUTO_REPAIR MUST_DEEP_SCRUB MUST_SCRUB planned REQ_SCRUB] scrubber: scrub_finish before flags: AUTO_REPAIR REQ_SCRUB. repair state: repair. deep_scrub_on_error: 0 2026-02-17T11:32:30.399 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:32:22.978+0000 7fd6270a4640 10 osd.1 pg_epoch: 45 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=44/45 n=10 ec=15/15 lis/c=44/44 les/c/f=45/45/0 sis=44) [1,0] r=0 lpr=44 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+repair [ 1.0: AUTO_REPAIR REQ_SCRUB ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-17T11:32:30.399 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:32:22.978+0000 7fd6270a4640 15 osd.1 pg_epoch: 45 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=44/45 n=9 ec=15/15 lis/c=44/44 les/c/f=45/45/0 sis=44) [1,0] r=0 lpr=44 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+repair [ 1.0: AUTO_REPAIR REQ_SCRUB ] ] scrubber: scrub_finish: 1 errors. 1 errors fixed 2026-02-17T11:32:30.399 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:32:22.978+0000 7fd6270a4640 20 osd.1 pg_epoch: 45 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=44/45 n=9 ec=15/15 lis/c=44/44 les/c/f=45/45/0 sis=44) [1,0] r=0 lpr=44 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+repair [ 1.0: AUTO_REPAIR REQ_SCRUB ] ] scrubber: scrub_finish All may be fixed 2026-02-17T11:32:30.399 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:32:22.978+0000 7fd6270a4640 19 osd.1 pg_epoch: 45 pg[1.0( v 19'10 (0'0,19'10] local-lis/les=44/45 n=9 ec=15/15 lis/c=44/44 les/c/f=45/45/0 sis=44) [1,0] r=0 lpr=44 crt=19'10 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+repair [ 1.0: AUTO_REPAIR REQ_SCRUB ] ] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T11:32:30.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T11:32:30.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:32:30.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:32:30.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:32:30.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:32:30.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:32:30.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:32:30.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:32:30.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:32:30.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:32:30.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:32:30.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:32:30.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:32:30.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:32:30.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:32:30.715 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:32:30.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:32:30.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:32:30.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:32:30.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:32:30.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:32:30.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:32:30.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:32:30.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:32:30.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:32:30.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:32:30.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:32:30.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:32:30.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:32:30.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T11:32:30.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T11:32:30.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T11:32:30.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T11:32:30.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:32:30.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:32:30.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:32:30.737 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:32:30.737 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:32:30.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:32:30.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:32:30.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:32:30.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:32:30.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:32:30.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:32:30.740 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:32:30.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:32:30.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:32:30.742 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:32:30.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:32:30.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:32:30.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:32:30.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:32:30.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:32:30.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:32:30.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:32:30.747 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:32:30.747 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:32:30.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:32:30.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:32:30.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:32:30.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:32:30.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T11:32:30.750 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:32:30.750 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:32:30.751 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:32:30.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T11:32:30.752 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:32:30.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T11:32:30.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:32:30.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T11:32:30.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_bluestore_failed_norecov td/osd-scrub-repair 2026-02-17T11:32:30.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:604: TEST_auto_repair_bluestore_failed_norecov: local dir=td/osd-scrub-repair 2026-02-17T11:32:30.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:605: TEST_auto_repair_bluestore_failed_norecov: local poolname=testpool 2026-02-17T11:32:30.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:608: TEST_auto_repair_bluestore_failed_norecov: run_mon td/osd-scrub-repair a 2026-02-17T11:32:30.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T11:32:30.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T11:32:30.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T11:32:30.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T11:32:30.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T11:32:30.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T11:32:30.774 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T11:32:30.774 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:32:30.774 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:32:30.774 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:32:30.774 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:32:30.774 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:32:30.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:32:30.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T11:32:30.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T11:32:30.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T11:32:30.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:32:30.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:32:30.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T11:32:30.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T11:32:30.804 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:32:30.804 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:32:30.804 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:32:30.804 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:32:30.804 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:32:30.804 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:32:30.804 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:32:30.805 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:32:30.805 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T11:32:30.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T11:32:30.861 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:32:30.861 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:32:30.861 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T11:32:30.861 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T11:32:30.862 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:32:30.862 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:32:30.862 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:32:30.863 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:32:30.863 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:32:30.863 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:32:30.863 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:32:30.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:32:30.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T11:32:30.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:609: TEST_auto_repair_bluestore_failed_norecov: run_mgr td/osd-scrub-repair x 2026-02-17T11:32:30.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T11:32:30.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T11:32:30.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T11:32:30.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T11:32:30.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T11:32:30.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T11:32:31.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T11:32:31.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:32:31.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:32:31.009 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:32:31.009 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:32:31.009 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:32:31.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:32:31.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:32:31.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:32:31.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:612: TEST_auto_repair_bluestore_failed_norecov: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:32:31.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:613: TEST_auto_repair_bluestore_failed_norecov: seq 0 2 2026-02-17T11:32:31.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:613: TEST_auto_repair_bluestore_failed_norecov: for id in $(seq 0 2) 2026-02-17T11:32:31.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:614: TEST_auto_repair_bluestore_failed_norecov: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:32:31.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:32:31.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:32:31.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T11:32:31.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:32:31.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:32:31.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:32:31.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:32:31.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:32:31.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:32:31.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:32:31.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:32:31.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:32:31.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:32:31.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:32:31.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:32:31.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:32:31.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:32:31.033 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:32:31.034 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:32:31.034 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:32:31.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:32:31.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:32:31.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:32:31.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:32:31.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:32:31.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:32:31.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:32:31.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:32:31.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:32:31.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:32:31.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:32:31.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:32:31.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:32:31.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:32:31.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:32:31.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=6396bbf9-fdd1-45b2-a066-99053dc1ccfe 2026-02-17T11:32:31.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 6396bbf9-fdd1-45b2-a066-99053dc1ccfe' 2026-02-17T11:32:31.037 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 6396bbf9-fdd1-45b2-a066-99053dc1ccfe 2026-02-17T11:32:31.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:32:31.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDPUZRpOuKdAhAAf6A0BX0masK2W/QWF/F5Fw== 2026-02-17T11:32:31.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDPUZRpOuKdAhAAf6A0BX0masK2W/QWF/F5Fw=="}' 2026-02-17T11:32:31.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 6396bbf9-fdd1-45b2-a066-99053dc1ccfe -i td/osd-scrub-repair/0/new.json 2026-02-17T11:32:31.138 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:32:31.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T11:32:31.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDPUZRpOuKdAhAAf6A0BX0masK2W/QWF/F5Fw== --osd-uuid 6396bbf9-fdd1-45b2-a066-99053dc1ccfe 2026-02-17T11:32:31.164 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:31.161+0000 7fedc19208c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:31.166 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:31.161+0000 7fedc19208c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:31.167 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:31.165+0000 7fedc19208c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:31.167 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:31.165+0000 7fedc19208c0 -1 bdev(0x5585a9dbfc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:32:31.167 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:31.165+0000 7fedc19208c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T11:32:33.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T11:32:33.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:32:33.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T11:32:33.733 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:32:33.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:32:33.925 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:32:33.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T11:32:33.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:32:33.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:32:33.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:32:33.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:32:33.949 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:33.945+0000 7f8b4f3268c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:33.951 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:33.949+0000 7f8b4f3268c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:33.952 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:33.949+0000 7f8b4f3268c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:34.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T11:32:34.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:32:34.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:32:34.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:32:34.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:32:34.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:34.079 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:32:34.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:32:34.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:34.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:32:34.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:35.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:35.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:35.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:32:35.230 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:32:35.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:35.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:32:35.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:35.440 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:35.437+0000 7f8b4f3268c0 -1 Falling back to public interface 2026-02-17T11:32:36.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:36.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:36.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:32:36.383 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:32:36.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:36.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:32:36.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:37.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:37.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:37.534 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:32:37.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:32:37.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:37.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:32:37.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:38.421 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:38.417+0000 7f8b4f3268c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:32:38.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:38.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:38.686 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:32:38.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:32:38.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:38.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:32:38.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:39.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:39.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:39.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:32:39.843 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:32:39.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:39.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:32:39.993 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2533466716,v1:127.0.0.1:6803/2533466716] [v2:127.0.0.1:6804/2533466716,v1:127.0.0.1:6805/2533466716] exists,up 6396bbf9-fdd1-45b2-a066-99053dc1ccfe 2026-02-17T11:32:39.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:32:39.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:32:39.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:32:39.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:613: TEST_auto_repair_bluestore_failed_norecov: for id in $(seq 0 2) 2026-02-17T11:32:39.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:614: TEST_auto_repair_bluestore_failed_norecov: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:32:39.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:32:39.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:32:39.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T11:32:39.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:32:39.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:32:39.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:32:39.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:32:39.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:32:39.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:32:39.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:32:39.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:32:39.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:32:39.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:32:39.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:32:39.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:32:39.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:32:39.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:32:39.999 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:32:39.999 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:32:39.999 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:32:39.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:32:39.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:32:39.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:32:39.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:32:40.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:32:40.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:32:40.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:32:40.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:32:40.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:32:40.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:32:40.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:32:40.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:32:40.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:32:40.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:32:40.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:32:40.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e54ff825-b084-42a4-9148-afd02c868acf 2026-02-17T11:32:40.002 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 e54ff825-b084-42a4-9148-afd02c868acf 2026-02-17T11:32:40.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 e54ff825-b084-42a4-9148-afd02c868acf' 2026-02-17T11:32:40.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:32:40.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDYUZRp745/ABAA6r2omCaBy/MMCW4qIrhEZw== 2026-02-17T11:32:40.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDYUZRp745/ABAA6r2omCaBy/MMCW4qIrhEZw=="}' 2026-02-17T11:32:40.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e54ff825-b084-42a4-9148-afd02c868acf -i td/osd-scrub-repair/1/new.json 2026-02-17T11:32:40.165 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:32:40.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T11:32:40.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDYUZRp745/ABAA6r2omCaBy/MMCW4qIrhEZw== --osd-uuid e54ff825-b084-42a4-9148-afd02c868acf 2026-02-17T11:32:40.192 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:40.189+0000 7fbb5967a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:40.193 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:40.189+0000 7fbb5967a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:40.194 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:40.189+0000 7fbb5967a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:40.195 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:40.189+0000 7fbb5967a8c0 -1 bdev(0x5571f9d7dc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:32:40.195 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:40.193+0000 7fbb5967a8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T11:32:42.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T11:32:42.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:32:42.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T11:32:42.473 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:32:42.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:32:42.656 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:32:42.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T11:32:42.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:32:42.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:32:42.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:32:42.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:32:42.680 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:42.677+0000 7f55d46948c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:42.681 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:42.677+0000 7f55d46948c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:42.682 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:42.677+0000 7f55d46948c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:42.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T11:32:42.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:32:42.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:32:42.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:32:42.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:32:42.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:42.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:32:42.810 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:32:42.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:42.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:32:42.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:43.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:43.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:43.959 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:32:43.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:32:43.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:43.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:32:44.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:44.168 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:44.165+0000 7f55d46948c0 -1 Falling back to public interface 2026-02-17T11:32:45.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:45.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:45.108 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:32:45.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:32:45.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:45.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:32:45.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:46.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:46.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:46.257 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:32:46.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:32:46.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:46.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:32:46.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:47.146 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:47.141+0000 7f55d46948c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:32:47.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:47.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:47.409 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:32:47.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:32:47.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:47.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:32:47.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:48.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:48.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:48.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:32:48.561 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:32:48.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:48.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:32:48.709 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1968107016,v1:127.0.0.1:6811/1968107016] [v2:127.0.0.1:6812/1968107016,v1:127.0.0.1:6813/1968107016] exists,up e54ff825-b084-42a4-9148-afd02c868acf 2026-02-17T11:32:48.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:32:48.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:32:48.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:613: TEST_auto_repair_bluestore_failed_norecov: for id in $(seq 0 2) 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:614: TEST_auto_repair_bluestore_failed_norecov: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:32:48.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:32:48.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:32:48.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:32:48.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:32:48.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:32:48.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:32:48.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:32:48.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:32:48.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:32:48.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:32:48.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:32:48.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:32:48.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:32:48.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:32:48.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:32:48.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=c3b5cbc4-93ec-46e4-a031-430bdf468073 2026-02-17T11:32:48.714 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 c3b5cbc4-93ec-46e4-a031-430bdf468073 2026-02-17T11:32:48.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 c3b5cbc4-93ec-46e4-a031-430bdf468073' 2026-02-17T11:32:48.714 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:32:48.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDgUZRpAfYoKxAAeYySSwcVK8Rcf9CVr0RKNw== 2026-02-17T11:32:48.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDgUZRpAfYoKxAAeYySSwcVK8Rcf9CVr0RKNw=="}' 2026-02-17T11:32:48.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new c3b5cbc4-93ec-46e4-a031-430bdf468073 -i td/osd-scrub-repair/2/new.json 2026-02-17T11:32:48.882 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:32:48.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T11:32:48.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDgUZRpAfYoKxAAeYySSwcVK8Rcf9CVr0RKNw== --osd-uuid c3b5cbc4-93ec-46e4-a031-430bdf468073 2026-02-17T11:32:48.908 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:48.905+0000 7f3eb71338c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:48.910 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:48.905+0000 7f3eb71338c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:48.911 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:48.905+0000 7f3eb71338c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:48.911 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:48.909+0000 7f3eb71338c0 -1 bdev(0x5615428a1c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T11:32:48.911 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:48.909+0000 7f3eb71338c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T11:32:51.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T11:32:51.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:32:51.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T11:32:51.193 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T11:32:51.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:32:51.384 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T11:32:51.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T11:32:51.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:32:51.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:32:51.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:32:51.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:32:51.408 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:51.405+0000 7f3ca173d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:51.410 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:51.405+0000 7f3ca173d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:51.410 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:51.405+0000 7f3ca173d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:32:51.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T11:32:51.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:32:51.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:32:51.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:32:51.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:32:51.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:51.541 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:32:51.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:32:51.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:51.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:32:51.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:52.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:52.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:52.693 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:32:52.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:32:52.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:52.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:32:52.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:52.909 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:52.905+0000 7f3ca173d8c0 -1 Falling back to public interface 2026-02-17T11:32:53.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:53.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:53.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:32:53.840 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:32:53.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:53.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:32:53.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:54.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:54.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:54.990 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:32:54.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:32:54.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:54.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:32:55.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:55.882 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:32:55.877+0000 7f3ca173d8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T11:32:56.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:56.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:56.141 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:32:56.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:32:56.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:56.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:32:56.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:32:57.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:32:57.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:32:57.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:32:57.293 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:32:57.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:32:57.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:32:57.448 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1249098544,v1:127.0.0.1:6819/1249098544] [v2:127.0.0.1:6820/1249098544,v1:127.0.0.1:6821/1249098544] exists,up c3b5cbc4-93ec-46e4-a031-430bdf468073 2026-02-17T11:32:57.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:32:57.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:32:57.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:32:57.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:617: TEST_auto_repair_bluestore_failed_norecov: create_pool testpool 1 1 2026-02-17T11:32:57.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create testpool 1 1 2026-02-17T11:32:57.599 INFO:tasks.workunit.client.0.trial128.stderr:pool 'testpool' created 2026-02-17T11:32:57.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:32:58.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:618: TEST_auto_repair_bluestore_failed_norecov: ceph osd pool set testpool size 2 2026-02-17T11:32:58.803 INFO:tasks.workunit.client.0.trial128.stderr:set pool 1 size to 2 2026-02-17T11:32:58.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:619: TEST_auto_repair_bluestore_failed_norecov: wait_for_clean 2026-02-17T11:32:58.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:32:58.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:32:58.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:32:58.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:32:58.815 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:32:58.816 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:32:58.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:32:58.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:32:58.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:32:58.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:32:58.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:32:58.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:32:58.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:32:58.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:32:58.932 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:32:59.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:32:59.088 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:32:59.088 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:32:59.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:32:59.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:32:59.089 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:32:59.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T11:32:59.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T11:32:59.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T11:32:59.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:32:59.162 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:32:59.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-17T11:32:59.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-17T11:32:59.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705669' 2026-02-17T11:32:59.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:32:59.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:32:59.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574851 2026-02-17T11:32:59.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574851 2026-02-17T11:32:59.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705669 2-55834574851' 2026-02-17T11:32:59.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:32:59.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T11:32:59.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:32:59.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:32:59.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T11:32:59.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:32:59.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T11:32:59.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T11:32:59.310 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836485 2026-02-17T11:32:59.311 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:32:59.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-17T11:32:59.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:32:59.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-17T11:32:59.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:32:59.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:32:59.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-17T11:32:59.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:32:59.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-17T11:32:59.474 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705669 2026-02-17T11:32:59.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-17T11:32:59.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:32:59.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-17T11:32:59.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:32:59.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574851 2026-02-17T11:32:59.635 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:32:59.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:32:59.636 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574851 2026-02-17T11:32:59.636 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:32:59.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574851 2026-02-17T11:32:59.638 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574851 2026-02-17T11:32:59.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-17T11:32:59.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:32:59.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-17T11:32:59.801 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:32:59.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:32:59.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:32:59.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:32:59.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:32:59.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:32:59.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:32:59.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:32:59.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:32:59.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:32:59.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:33:00.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T11:33:00.145 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:33:00.145 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:33:00.146 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:33:00.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T11:33:00.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T11:33:00.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T11:33:00.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T11:33:00.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:33:00.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T11:33:00.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:33:00.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:33:00.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:33:00.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:33:00.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:33:00.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:33:00.437 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:33:00.437 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:33:00.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T11:33:00.591 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:33:00.591 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:33:00.591 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:33:00.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T11:33:00.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T11:33:00.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:33:00.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:33:00.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:33:00.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:33:00.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:33:00.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:33:00.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:33:00.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:33:00.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:33:00.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T11:33:00.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:33:00.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T11:33:01.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:33:01.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:33:01.167 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:33:01.167 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:33:01.167 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:33:01.167 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:33:01.167 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:33:01.167 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:33:01.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T11:33:01.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:33:01.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:33:01.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:33:01.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T11:33:01.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T11:33:01.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:33:01.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:33:01.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:33:01.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:33:01.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:33:01.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:33:01.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:33:01.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:33:01.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:33:01.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T11:33:01.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:33:01.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T11:33:02.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:33:02.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:33:02.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:33:02.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:33:02.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:33:02.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:33:02.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:33:02.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:33:02.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T11:33:02.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:33:02.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:33:02.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:33:02.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T11:33:02.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T11:33:02.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:33:02.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:33:02.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:33:02.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:33:02.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:33:02.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:33:02.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:33:02.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:33:02.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:33:02.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 13 )) 2026-02-17T11:33:02.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:33:02.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T11:33:03.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:33:03.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:33:03.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:33:03.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:33:03.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:33:03.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:33:03.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:33:03.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:33:03.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:33:03.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:33:03.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:33:03.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:33:03.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:33:03.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:33:03.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:33:03.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:622: TEST_auto_repair_bluestore_failed_norecov: local payload=ABCDEF 2026-02-17T11:33:03.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:623: TEST_auto_repair_bluestore_failed_norecov: echo ABCDEF 2026-02-17T11:33:03.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: seq 1 10 2026-02-17T11:33:03.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T11:33:03.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj1 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:33:03.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T11:33:03.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj2 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:33:03.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T11:33:03.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj3 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:33:03.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T11:33:03.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj4 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:33:03.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T11:33:03.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj5 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:33:03.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T11:33:03.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj6 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:33:03.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T11:33:03.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj7 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:33:03.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T11:33:03.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj8 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:33:03.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T11:33:03.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj9 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:33:03.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:624: TEST_auto_repair_bluestore_failed_norecov: for i in $(seq 1 10) 2026-02-17T11:33:03.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:626: TEST_auto_repair_bluestore_failed_norecov: rados --pool testpool put obj10 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:33:03.986 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:632: TEST_auto_repair_bluestore_failed_norecov: get_not_primary testpool SOMETHING 2026-02-17T11:33:03.987 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T11:33:03.987 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:33:03.988 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T11:33:03.988 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:33:03.988 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:33:03.988 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:33:03.988 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:33:04.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T11:33:04.134 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:33:04.134 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T11:33:04.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:632: TEST_auto_repair_bluestore_failed_norecov: objectstore_tool td/osd-scrub-repair 0 obj1 remove 2026-02-17T11:33:04.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:33:04.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:33:04.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:33:04.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:33:04.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 obj1 remove 2026-02-17T11:33:04.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:33:04.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:33:04.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:33:04.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:33:04.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:33:04.292 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:33:04.292 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:33:04.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:33:04.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:33:04.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:33:05.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:33:05.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 obj1 remove 2026-02-17T11:33:05.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:33:05.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:33:05.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:33:05.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:33:05.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:33:05.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj1 remove 2026-02-17T11:33:06.249 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-17T11:33:06.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:33:06.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:33:06.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:33:06.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:33:06.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:33:06.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:33:06.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:33:06.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:33:06.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:33:06.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:33:06.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:33:06.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:33:06.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:33:06.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:33:06.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:33:06.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:33:06.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:33:06.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:33:06.787 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:33:06.787 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:33:06.787 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:33:06.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:33:06.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:33:06.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:33:06.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:33:06.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:33:06.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:33:06.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:33:06.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:33:06.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:33:06.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:33:06.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:33:06.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:33:06.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:33:06.789 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:33:06.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:33:06.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:33:06.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:33:06.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:33:06.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:33:06.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:33:06.812 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:06.808+0000 7f24d4f428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:33:06.813 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:06.808+0000 7f24d4f428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:33:06.814 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:06.808+0000 7f24d4f428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:33:06.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:33:06.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:33:06.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:33:06.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:33:06.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:33:06.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:06.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:33:06.937 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:33:06.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:06.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:33:07.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:33:08.069 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:08.064+0000 7f24d4f428c0 -1 Falling back to public interface 2026-02-17T11:33:08.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:33:08.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:08.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:33:08.091 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:33:08.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:08.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:33:08.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:33:09.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:33:09.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:09.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:33:09.245 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:33:09.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:09.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:33:09.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:33:10.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:33:10.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:10.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:33:10.402 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:33:10.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:10.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:33:10.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:33:11.083 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:11.076+0000 7f24d4f428c0 -1 osd.0 19 log_to_monitors true 2026-02-17T11:33:11.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:33:11.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:11.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:33:11.557 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:33:11.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:11.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:33:11.711 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 23 up_thru 0 down_at 20 last_clean_interval [5,19) [v2:127.0.0.1:6802/1095111589,v1:127.0.0.1:6803/1095111589] [v2:127.0.0.1:6804/1095111589,v1:127.0.0.1:6805/1095111589] exists,up 6396bbf9-fdd1-45b2-a066-99053dc1ccfe 2026-02-17T11:33:11.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:33:11.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:33:11.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:33:11.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:33:11.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:33:11.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:33:11.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:33:11.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:33:11.713 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:33:11.713 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:33:11.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:33:11.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:33:11.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:33:11.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:33:11.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:33:11.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:33:11.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:33:11.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:33:11.820 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:33:11.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:33:11.979 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:33:11.979 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:33:11.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:33:11.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:33:11.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:33:12.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247810 2026-02-17T11:33:12.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247810 2026-02-17T11:33:12.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810' 2026-02-17T11:33:12.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:33:12.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:33:12.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705672 2026-02-17T11:33:12.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705672 2026-02-17T11:33:12.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810 1-38654705672' 2026-02-17T11:33:12.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:33:12.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:33:12.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574855 2026-02-17T11:33:12.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574855 2026-02-17T11:33:12.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810 1-38654705672 2-55834574855' 2026-02-17T11:33:12.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:33:12.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247810 2026-02-17T11:33:12.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:33:12.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:33:12.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247810 2026-02-17T11:33:12.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:33:12.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247810 2026-02-17T11:33:12.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247810' 2026-02-17T11:33:12.194 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 98784247810 2026-02-17T11:33:12.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:33:12.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 98784247810 2026-02-17T11:33:12.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:33:13.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:33:13.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:33:13.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247810 -lt 98784247810 2026-02-17T11:33:13.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:33:13.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705672 2026-02-17T11:33:13.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:33:13.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:33:13.512 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705672 2026-02-17T11:33:13.512 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:33:13.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705672 2026-02-17T11:33:13.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705672' 2026-02-17T11:33:13.514 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705672 2026-02-17T11:33:13.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:33:13.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705673 -lt 38654705672 2026-02-17T11:33:13.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:33:13.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574855 2026-02-17T11:33:13.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:33:13.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:33:13.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574855 2026-02-17T11:33:13.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:33:13.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574855 2026-02-17T11:33:13.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574855' 2026-02-17T11:33:13.677 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 55834574855 2026-02-17T11:33:13.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:33:13.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574855 -lt 55834574855 2026-02-17T11:33:13.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:33:13.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:33:13.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:33:14.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:33:14.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:33:14.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:33:14.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:33:14.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:33:14.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:33:14.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:33:14.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:33:14.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:33:14.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:33:14.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:33:14.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:33:14.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:33:14.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:33:14.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:33:14.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:633: TEST_auto_repair_bluestore_failed_norecov: get_primary testpool SOMETHING 2026-02-17T11:33:14.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:33:14.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:33:14.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:33:14.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:33:14.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:633: TEST_auto_repair_bluestore_failed_norecov: objectstore_tool td/osd-scrub-repair 1 obj1 rm-attr _ 2026-02-17T11:33:14.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:33:14.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:33:14.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:33:14.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:33:14.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 obj1 rm-attr _ 2026-02-17T11:33:14.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:33:14.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:33:14.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:33:14.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:33:14.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:33:14.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:33:14.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:33:14.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:33:14.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:33:14.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:33:15.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:33:15.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 obj1 rm-attr _ 2026-02-17T11:33:15.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:33:15.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:33:15.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:33:15.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:33:15.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:33:15.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj1 rm-attr _ 2026-02-17T11:33:17.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:33:17.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:33:17.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:33:17.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:33:17.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:33:17.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:33:17.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:33:17.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:33:17.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:33:17.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:33:17.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:33:17.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:33:17.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:33:17.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:33:17.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:33:17.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:33:17.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:33:17.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:33:17.022 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:33:17.022 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:33:17.022 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:33:17.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:33:17.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:33:17.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:33:17.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:33:17.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:33:17.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:33:17.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:33:17.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:33:17.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:33:17.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:33:17.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:33:17.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:33:17.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:33:17.024 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:33:17.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:33:17.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:33:17.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:33:17.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:33:17.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:33:17.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:33:17.044 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:17.040+0000 7f217963f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:33:17.046 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:17.040+0000 7f217963f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:33:17.046 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:17.040+0000 7f217963f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:33:17.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:33:17.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:33:17.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:33:17.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:33:17.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:33:17.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:17.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:33:17.181 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:33:17.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:17.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:33:17.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:33:18.269 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:18.264+0000 7f217963f8c0 -1 Falling back to public interface 2026-02-17T11:33:18.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:33:18.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:18.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:33:18.331 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:33:18.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:18.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:33:18.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:33:19.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:33:19.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:19.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:33:19.484 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:33:19.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:19.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:33:19.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:33:20.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:33:20.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:20.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:33:20.634 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:33:20.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:20.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:33:20.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:33:21.284 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:21.280+0000 7f217963f8c0 -1 osd.1 25 log_to_monitors true 2026-02-17T11:33:21.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:33:21.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:21.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:33:21.793 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:33:21.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:21.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:33:21.942 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 29 up_thru 29 down_at 26 last_clean_interval [9,25) [v2:127.0.0.1:6810/3620277892,v1:127.0.0.1:6811/3620277892] [v2:127.0.0.1:6812/3620277892,v1:127.0.0.1:6813/3620277892] exists,up e54ff825-b084-42a4-9148-afd02c868acf 2026-02-17T11:33:21.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:33:21.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:33:21.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:33:21.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:33:21.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:33:21.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:33:21.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:33:21.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:33:21.943 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:33:21.943 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:33:21.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:33:21.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:33:21.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:33:22.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:33:22.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:33:22.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:33:22.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:33:22.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:33:22.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:33:22.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:33:22.191 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:33:22.191 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:33:22.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:33:22.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:33:22.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:33:22.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247813 2026-02-17T11:33:22.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247813 2026-02-17T11:33:22.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247813' 2026-02-17T11:33:22.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:33:22.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:33:22.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051586 2026-02-17T11:33:22.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051586 2026-02-17T11:33:22.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247813 1-124554051586' 2026-02-17T11:33:22.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:33:22.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:33:22.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574858 2026-02-17T11:33:22.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574858 2026-02-17T11:33:22.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247813 1-124554051586 2-55834574858' 2026-02-17T11:33:22.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:33:22.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247813 2026-02-17T11:33:22.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:33:22.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:33:22.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247813 2026-02-17T11:33:22.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:33:22.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247813 2026-02-17T11:33:22.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247813' 2026-02-17T11:33:22.415 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 98784247813 2026-02-17T11:33:22.415 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:33:22.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247812 -lt 98784247813 2026-02-17T11:33:22.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:33:23.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:33:23.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:33:23.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247813 -lt 98784247813 2026-02-17T11:33:23.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:33:23.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-124554051586 2026-02-17T11:33:23.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:33:23.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:33:23.732 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-124554051586 2026-02-17T11:33:23.732 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:33:23.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051586 2026-02-17T11:33:23.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 124554051586' 2026-02-17T11:33:23.734 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 124554051586 2026-02-17T11:33:23.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:33:23.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051586 -lt 124554051586 2026-02-17T11:33:23.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:33:23.892 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574858 2026-02-17T11:33:23.892 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:33:23.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:33:23.893 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574858 2026-02-17T11:33:23.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:33:23.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574858 2026-02-17T11:33:23.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574858' 2026-02-17T11:33:23.895 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 55834574858 2026-02-17T11:33:23.896 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:33:24.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574858 -lt 55834574858 2026-02-17T11:33:24.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:33:24.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:33:24.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:33:24.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:33:24.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:33:24.243 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:33:24.243 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:33:24.243 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:33:24.243 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:33:24.243 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:33:24.243 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:33:24.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:33:24.396 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:33:24.396 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:33:24.397 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:33:24.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:33:24.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:33:24.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:33:24.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:635: TEST_auto_repair_bluestore_failed_norecov: get_not_primary testpool SOMETHING 2026-02-17T11:33:24.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T11:33:24.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:33:24.587 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T11:33:24.587 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:33:24.587 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:33:24.587 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:33:24.587 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:33:24.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T11:33:24.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:33:24.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T11:33:24.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:635: TEST_auto_repair_bluestore_failed_norecov: objectstore_tool td/osd-scrub-repair 0 obj2 remove 2026-02-17T11:33:24.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:33:24.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:33:24.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:33:24.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:33:24.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 obj2 remove 2026-02-17T11:33:24.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:33:24.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:33:24.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:33:24.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:33:24.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:33:24.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:33:24.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:33:24.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:33:24.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:33:24.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:33:26.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:33:26.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 obj2 remove 2026-02-17T11:33:26.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:33:26.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:33:26.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:33:26.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:33:26.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:33:26.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj2 remove 2026-02-17T11:33:26.837 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:104778fc:::obj2:head# 2026-02-17T11:33:27.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:33:27.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:33:27.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:33:27.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:33:27.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:33:27.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:33:27.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:33:27.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:33:27.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:33:27.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:33:27.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:33:27.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:33:27.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:33:27.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:33:27.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:33:27.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:33:27.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:33:27.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:33:27.384 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:33:27.384 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:33:27.384 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:33:27.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:33:27.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:33:27.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:33:27.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:33:27.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:33:27.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:33:27.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:33:27.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:33:27.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:33:27.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:33:27.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:33:27.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:33:27.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:33:27.388 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:33:27.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:33:27.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:33:27.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:33:27.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:33:27.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:33:27.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:33:27.405 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:27.400+0000 7fccfaab38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:33:27.406 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:27.400+0000 7fccfaab38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:33:27.407 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:27.400+0000 7fccfaab38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:33:27.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:33:27.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:33:27.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:33:27.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:33:27.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:33:27.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:27.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:33:27.539 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:33:27.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:27.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:33:27.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:33:28.646 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:28.640+0000 7fccfaab38c0 -1 Falling back to public interface 2026-02-17T11:33:28.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:33:28.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:28.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:33:28.691 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:33:28.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:28.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:33:28.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:33:29.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:33:29.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:29.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:33:29.842 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:33:29.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:29.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:33:29.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:33:30.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:33:31.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:31.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:33:31.000 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:33:31.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:31.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:33:31.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:33:31.617 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:31.611+0000 7fccfaab38c0 -1 osd.0 31 log_to_monitors true 2026-02-17T11:33:32.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:33:32.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:32.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:33:32.153 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:33:32.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:32.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:33:32.299 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 35 up_thru 26 down_at 32 last_clean_interval [23,31) [v2:127.0.0.1:6802/3469107814,v1:127.0.0.1:6803/3469107814] [v2:127.0.0.1:6804/3469107814,v1:127.0.0.1:6805/3469107814] exists,up 6396bbf9-fdd1-45b2-a066-99053dc1ccfe 2026-02-17T11:33:32.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:33:32.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:33:32.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:33:32.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:33:32.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:33:32.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:33:32.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:33:32.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:33:32.301 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:33:32.301 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:33:32.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:33:32.302 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:33:32.302 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:33:32.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:33:32.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:33:32.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:33:32.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:33:32.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:33:32.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:33:32.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:33:32.560 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:33:32.560 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:33:32.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:33:32.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:33:32.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:33:32.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855362 2026-02-17T11:33:32.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855362 2026-02-17T11:33:32.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855362' 2026-02-17T11:33:32.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:33:32.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:33:32.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051589 2026-02-17T11:33:32.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051589 2026-02-17T11:33:32.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855362 1-124554051589' 2026-02-17T11:33:32.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:33:32.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:33:32.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574861 2026-02-17T11:33:32.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574861 2026-02-17T11:33:32.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855362 1-124554051589 2-55834574861' 2026-02-17T11:33:32.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:33:32.782 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855362 2026-02-17T11:33:32.782 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:33:32.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:33:32.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855362 2026-02-17T11:33:32.784 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:33:32.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855362 2026-02-17T11:33:32.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855362' 2026-02-17T11:33:32.785 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 150323855362 2026-02-17T11:33:32.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:33:32.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247813 -lt 150323855362 2026-02-17T11:33:32.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:33:33.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:33:33.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:33:34.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855362 -lt 150323855362 2026-02-17T11:33:34.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:33:34.101 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-124554051589 2026-02-17T11:33:34.101 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:33:34.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:33:34.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-124554051589 2026-02-17T11:33:34.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:33:34.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051589 2026-02-17T11:33:34.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 124554051589' 2026-02-17T11:33:34.105 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 124554051589 2026-02-17T11:33:34.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:33:34.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051589 -lt 124554051589 2026-02-17T11:33:34.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:33:34.260 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574861 2026-02-17T11:33:34.261 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:33:34.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:33:34.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574861 2026-02-17T11:33:34.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:33:34.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574861 2026-02-17T11:33:34.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574861' 2026-02-17T11:33:34.264 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 55834574861 2026-02-17T11:33:34.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:33:34.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574861 -lt 55834574861 2026-02-17T11:33:34.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:33:34.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:33:34.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:33:34.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:33:34.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:33:34.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:33:34.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:33:34.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:33:34.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:33:34.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:33:34.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:33:34.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:33:34.746 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:33:34.747 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:33:34.747 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:33:34.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:33:34.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:33:34.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:33:34.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:636: TEST_auto_repair_bluestore_failed_norecov: get_primary testpool SOMETHING 2026-02-17T11:33:34.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:33:34.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:33:34.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:33:34.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:33:35.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:636: TEST_auto_repair_bluestore_failed_norecov: objectstore_tool td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-17T11:33:35.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:33:35.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:33:35.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:33:35.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:33:35.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-17T11:33:35.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:33:35.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:33:35.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:33:35.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:33:35.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:33:35.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:33:35.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:33:35.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:33:35.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:33:35.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:33:36.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:33:36.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 obj2 rm-attr _ 2026-02-17T11:33:36.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:33:36.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:33:36.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:33:36.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:33:36.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:33:36.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj2 rm-attr _ 2026-02-17T11:33:37.049 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:ff7b1f36:::obj1:head#, (61) No data available 2026-02-17T11:33:37.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:33:37.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:33:37.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:33:37.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:33:37.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:33:37.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:33:37.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:33:37.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:33:37.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:33:37.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:33:37.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:33:37.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:33:37.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:33:37.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:33:37.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:33:37.591 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:33:37.591 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:33:37.591 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:33:37.591 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:33:37.591 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:33:37.591 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:33:37.591 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:33:37.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:33:37.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:33:37.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:33:37.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:33:37.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:33:37.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:33:37.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:33:37.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:33:37.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:33:37.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:33:37.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:33:37.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:33:37.593 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:33:37.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:33:37.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:33:37.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:33:37.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:33:37.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:33:37.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:33:37.613 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:37.607+0000 7f7ae2aec8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:33:37.614 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:37.607+0000 7f7ae2aec8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:33:37.615 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:37.607+0000 7f7ae2aec8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:33:37.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:33:37.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:33:37.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:33:37.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:33:37.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:33:37.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:37.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:33:37.747 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:33:37.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:37.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:33:37.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:33:38.869 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:38.863+0000 7f7ae2aec8c0 -1 Falling back to public interface 2026-02-17T11:33:38.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:33:38.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:38.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:33:38.904 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:33:38.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:38.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:33:39.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:33:40.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:33:40.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:40.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:33:40.056 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:33:40.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:40.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:33:40.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:33:41.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:33:41.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:41.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:33:41.203 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:33:41.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:41.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:33:41.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:33:41.857 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:33:41.851+0000 7f7ae2aec8c0 -1 osd.1 36 log_to_monitors true 2026-02-17T11:33:42.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:33:42.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:33:42.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:33:42.355 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:33:42.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:33:42.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:33:42.505 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 40 up_thru 40 down_at 37 last_clean_interval [29,36) [v2:127.0.0.1:6810/2098951819,v1:127.0.0.1:6811/2098951819] [v2:127.0.0.1:6812/2098951819,v1:127.0.0.1:6813/2098951819] exists,up e54ff825-b084-42a4-9148-afd02c868acf 2026-02-17T11:33:42.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:33:42.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:33:42.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:33:42.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:33:42.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:33:42.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:33:42.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:33:42.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:33:42.506 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:33:42.506 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:33:42.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:33:42.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:33:42.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:33:42.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:33:42.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:33:42.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:33:42.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:33:42.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:33:42.614 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:33:42.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:33:42.768 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:33:42.768 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:33:42.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:33:42.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:33:42.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:33:42.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855365 2026-02-17T11:33:42.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855365 2026-02-17T11:33:42.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855365' 2026-02-17T11:33:42.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:33:42.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:33:42.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691842 2026-02-17T11:33:42.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691842 2026-02-17T11:33:42.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855365 1-171798691842' 2026-02-17T11:33:42.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:33:42.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:33:42.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574864 2026-02-17T11:33:42.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574864 2026-02-17T11:33:42.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855365 1-171798691842 2-55834574864' 2026-02-17T11:33:42.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:33:42.986 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855365 2026-02-17T11:33:42.987 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:33:42.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:33:42.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855365 2026-02-17T11:33:42.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:33:42.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855365 2026-02-17T11:33:42.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855365' 2026-02-17T11:33:42.990 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 150323855365 2026-02-17T11:33:42.990 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:33:43.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855363 -lt 150323855365 2026-02-17T11:33:43.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:33:44.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:33:44.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:33:44.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855365 -lt 150323855365 2026-02-17T11:33:44.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:33:44.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-171798691842 2026-02-17T11:33:44.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:33:44.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:33:44.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-171798691842 2026-02-17T11:33:44.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:33:44.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691842 2026-02-17T11:33:44.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 171798691842' 2026-02-17T11:33:44.308 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 171798691842 2026-02-17T11:33:44.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:33:44.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691842 -lt 171798691842 2026-02-17T11:33:44.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:33:44.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574864 2026-02-17T11:33:44.474 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:33:44.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:33:44.476 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574864 2026-02-17T11:33:44.476 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:33:44.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574864 2026-02-17T11:33:44.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574864' 2026-02-17T11:33:44.477 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 55834574864 2026-02-17T11:33:44.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:33:44.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574864 -lt 55834574864 2026-02-17T11:33:44.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:33:44.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:33:44.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:33:44.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:33:44.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:33:44.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:33:44.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:33:44.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:33:44.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:33:44.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:33:44.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:33:44.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:33:44.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:33:44.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:33:44.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:33:45.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:33:45.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:33:45.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:33:45.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:637: TEST_auto_repair_bluestore_failed_norecov: ceph tell 'osd.*' config set osd_scrub_auto_repair true 2026-02-17T11:33:45.231 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T11:33:45.231 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_auto_repair = '' (not observed, change may require restart) osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-17T11:33:45.231 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:33:45.238 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T11:33:45.238 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_auto_repair = '' (not observed, change may require restart) osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-17T11:33:45.238 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:33:45.244 INFO:tasks.workunit.client.0.trial128.stdout:osd.2: { 2026-02-17T11:33:45.244 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_auto_repair = '' (not observed, change may require restart) " 2026-02-17T11:33:45.244 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:33:45.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:639: TEST_auto_repair_bluestore_failed_norecov: get_pg testpool obj1 2026-02-17T11:33:45.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=testpool 2026-02-17T11:33:45.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=obj1 2026-02-17T11:33:45.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map testpool obj1 2026-02-17T11:33:45.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:33:45.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:639: TEST_auto_repair_bluestore_failed_norecov: local pgid=1.0 2026-02-17T11:33:45.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:640: TEST_auto_repair_bluestore_failed_norecov: get_primary testpool obj1 2026-02-17T11:33:45.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:33:45.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T11:33:45.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool obj1 2026-02-17T11:33:45.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:33:45.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:640: TEST_auto_repair_bluestore_failed_norecov: local primary=1 2026-02-17T11:33:45.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:641: TEST_auto_repair_bluestore_failed_norecov: get_last_scrub_stamp 1.0 2026-02-17T11:33:45.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:33:45.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:33:45.562 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:33:45.562 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:33:45.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:641: TEST_auto_repair_bluestore_failed_norecov: local last_scrub_stamp=2026-02-17T11:32:57.595439+0000 2026-02-17T11:33:45.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:642: TEST_auto_repair_bluestore_failed_norecov: ceph tell 1.0 schedule-deep-scrub 2026-02-17T11:33:45.775 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T11:33:45.775 INFO:tasks.workunit.client.0.trial128.stdout: "deep": true, 2026-02-17T11:33:45.775 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T11:33:45.776 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-03T11:32:05.772549+0000" 2026-02-17T11:33:45.776 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:33:45.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:645: TEST_auto_repair_bluestore_failed_norecov: wait_for_scrub 1.0 2026-02-17T11:32:57.595439+0000 2026-02-17T11:33:45.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T11:33:45.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:32:57.595439+0000 2026-02-17T11:33:45.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:33:45.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:33:45.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:33:45.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:33:45.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:33:45.786 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:33:45.786 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:33:45.786 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:33:45.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:32:57.595439+0000 '>' 2026-02-17T11:32:57.595439+0000 2026-02-17T11:33:45.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:33:46.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:33:46.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:33:46.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:33:46.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:33:46.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:33:46.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:33:46.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:33:47.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:32:57.595439+0000 '>' 2026-02-17T11:32:57.595439+0000 2026-02-17T11:33:47.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:33:48.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:33:48.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:33:48.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:33:48.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:33:48.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:33:48.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:33:48.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:33:48.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T11:32:05.772549+0000 '>' 2026-02-17T11:32:57.595439+0000 2026-02-17T11:33:48.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:33:49.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:33:49.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:33:49.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:33:49.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:33:49.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:33:49.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:33:49.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:33:49.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T11:32:05.772549+0000 '>' 2026-02-17T11:32:57.595439+0000 2026-02-17T11:33:49.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:33:50.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:33:50.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:33:50.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:33:50.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:33:50.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:33:50.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:33:50.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:33:50.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T11:32:05.772549+0000 '>' 2026-02-17T11:32:57.595439+0000 2026-02-17T11:33:50.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:33:51.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:33:51.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:33:51.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:33:51.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:33:51.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:33:51.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:33:51.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:33:51.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T11:32:05.772549+0000 '>' 2026-02-17T11:32:57.595439+0000 2026-02-17T11:33:51.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:33:52.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:33:52.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:33:52.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:33:52.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:33:52.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:33:52.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:33:52.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:33:52.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T11:32:05.772549+0000 '>' 2026-02-17T11:32:57.595439+0000 2026-02-17T11:33:52.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:33:53.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:33:53.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:33:53.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:33:53.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:33:53.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:33:53.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:33:53.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:33:54.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T11:32:05.772549+0000 '>' 2026-02-17T11:32:57.595439+0000 2026-02-17T11:33:54.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:33:55.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:33:55.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:33:55.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:33:55.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:33:55.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:33:55.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:33:55.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:33:55.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T11:32:05.772549+0000 '>' 2026-02-17T11:32:57.595439+0000 2026-02-17T11:33:55.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:33:56.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:33:56.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:33:56.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:33:56.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:33:56.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:33:56.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:33:56.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:33:56.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T11:32:05.772549+0000 '>' 2026-02-17T11:32:57.595439+0000 2026-02-17T11:33:56.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:33:57.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:33:57.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:33:57.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:33:57.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:33:57.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:33:57.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:33:57.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:33:57.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:33:52.843162+0000 '>' 2026-02-17T11:32:57.595439+0000 2026-02-17T11:33:57.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:33:57.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:646: TEST_auto_repair_bluestore_failed_norecov: wait_for_clean 2026-02-17T11:33:57.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:33:57.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:33:57.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:33:57.458 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:33:57.458 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:33:57.458 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:33:57.458 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:33:57.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:33:57.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:33:57.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:33:57.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:33:57.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:33:57.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:33:57.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:33:57.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:33:57.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:33:57.722 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:33:57.722 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:33:57.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:33:57.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:33:57.723 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:33:57.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855369 2026-02-17T11:33:57.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855369 2026-02-17T11:33:57.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855369' 2026-02-17T11:33:57.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:33:57.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:33:57.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691846 2026-02-17T11:33:57.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691846 2026-02-17T11:33:57.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855369 1-171798691846' 2026-02-17T11:33:57.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:33:57.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:33:57.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574868 2026-02-17T11:33:57.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574868 2026-02-17T11:33:57.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855369 1-171798691846 2-55834574868' 2026-02-17T11:33:57.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:33:57.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855369 2026-02-17T11:33:57.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:33:57.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:33:57.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855369 2026-02-17T11:33:57.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:33:57.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855369 2026-02-17T11:33:57.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855369' 2026-02-17T11:33:57.942 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 150323855369 2026-02-17T11:33:57.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:33:58.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855368 -lt 150323855369 2026-02-17T11:33:58.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:33:59.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:33:59.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:33:59.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855368 -lt 150323855369 2026-02-17T11:33:59.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:34:00.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:34:00.261 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:34:00.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855369 -lt 150323855369 2026-02-17T11:34:00.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:34:00.424 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-171798691846 2026-02-17T11:34:00.424 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:34:00.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:34:00.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-171798691846 2026-02-17T11:34:00.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:34:00.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691846 2026-02-17T11:34:00.427 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 171798691846 2026-02-17T11:34:00.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 171798691846' 2026-02-17T11:34:00.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:34:00.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691846 -lt 171798691846 2026-02-17T11:34:00.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:34:00.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574868 2026-02-17T11:34:00.590 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:34:00.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:34:00.592 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574868 2026-02-17T11:34:00.592 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:34:00.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574868 2026-02-17T11:34:00.593 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574868 2026-02-17T11:34:00.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574868' 2026-02-17T11:34:00.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:34:00.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574868 -lt 55834574868 2026-02-17T11:34:00.755 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:34:00.755 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:34:00.755 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:34:00.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:34:00.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:34:00.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:34:00.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:34:00.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:34:00.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:34:00.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:34:00.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:34:01.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:34:01.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:34:01.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:34:01.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:34:01.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:34:01.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:34:01.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:34:01.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:647: TEST_auto_repair_bluestore_failed_norecov: flush_pg_stats 2026-02-17T11:34:01.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:34:01.261 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:34:01.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:34:01.414 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:34:01.414 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:34:01.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:34:01.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:34:01.415 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:34:01.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855370 2026-02-17T11:34:01.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855370 2026-02-17T11:34:01.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855370' 2026-02-17T11:34:01.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:34:01.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:34:01.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691847 2026-02-17T11:34:01.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691847 2026-02-17T11:34:01.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855370 1-171798691847' 2026-02-17T11:34:01.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:34:01.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:34:01.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574869 2026-02-17T11:34:01.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574869 2026-02-17T11:34:01.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855370 1-171798691847 2-55834574869' 2026-02-17T11:34:01.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:34:01.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855370 2026-02-17T11:34:01.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:34:01.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:34:01.632 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855370 2026-02-17T11:34:01.632 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:34:01.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855370 2026-02-17T11:34:01.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855370' 2026-02-17T11:34:01.634 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 150323855370 2026-02-17T11:34:01.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:34:01.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855369 -lt 150323855370 2026-02-17T11:34:01.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:34:02.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:34:02.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:34:02.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855369 -lt 150323855370 2026-02-17T11:34:02.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:34:03.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:34:03.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:34:04.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855371 -lt 150323855370 2026-02-17T11:34:04.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:34:04.101 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-171798691847 2026-02-17T11:34:04.101 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:34:04.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:34:04.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-171798691847 2026-02-17T11:34:04.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:34:04.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691847 2026-02-17T11:34:04.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 171798691847' 2026-02-17T11:34:04.105 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 171798691847 2026-02-17T11:34:04.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:34:04.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691848 -lt 171798691847 2026-02-17T11:34:04.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:34:04.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574869 2026-02-17T11:34:04.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:34:04.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:34:04.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574869 2026-02-17T11:34:04.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:34:04.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574869 2026-02-17T11:34:04.267 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574869 2026-02-17T11:34:04.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574869' 2026-02-17T11:34:04.267 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:34:04.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574870 -lt 55834574869 2026-02-17T11:34:04.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:648: TEST_auto_repair_bluestore_failed_norecov: grep -q 'scrub_finish.*present with no repair possible' td/osd-scrub-repair/osd.1.log 2026-02-17T11:34:04.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:649: TEST_auto_repair_bluestore_failed_norecov: ceph pg dump pgs 2026-02-17T11:34:04.573 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T11:34:04.573 INFO:tasks.workunit.client.0.trial128.stdout:1.0 10 0 0 0 0 56 0 0 10 0 10 active+clean+inconsistent+failed_repair 2026-02-17T11:33:52.843229+0000 19'10 41:95 [1,0] 1 [1,0] 1 19'10 2026-02-17T11:33:52.843162+0000 19'10 2026-02-17T11:33:52.843162+0000 0 1 periodic scrub scheduled @ 2026-02-18T11:33:52.843162+0000 10 0 2026-02-17T11:34:04.573 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:34:04.573 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T11:34:04.574 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:34:04.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:650: TEST_auto_repair_bluestore_failed_norecov: ceph pg dump pgs 2026-02-17T11:34:04.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:650: TEST_auto_repair_bluestore_failed_norecov: grep -q '^1.0.*+failed_repair' 2026-02-17T11:34:04.738 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:34:04.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T11:34:04.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:34:04.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:34:04.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:34:04.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:34:04.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:34:04.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:34:04.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:34:04.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:34:04.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:34:04.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:34:04.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:34:04.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:34:04.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:34:04.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:34:04.862 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:34:04.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:34:04.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:34:04.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:34:04.864 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:34:04.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:34:04.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:34:04.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:34:04.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:34:04.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:34:04.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:34:04.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:34:04.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:34:04.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:34:04.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T11:34:04.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T11:34:04.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T11:34:04.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T11:34:04.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:34:04.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:34:04.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:34:04.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:34:04.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:34:04.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:34:04.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:34:04.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:34:04.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:34:04.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:34:04.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:34:04.888 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:34:04.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:34:04.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:34:04.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:34:04.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:34:04.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:34:04.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:34:04.892 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:34:04.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:34:04.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:34:04.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:34:04.896 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:34:04.896 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:34:04.896 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:34:04.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:34:04.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:34:04.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:34:04.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T11:34:04.898 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:34:04.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:34:04.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:34:04.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T11:34:04.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:34:04.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T11:34:04.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:34:04.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T11:34:04.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_bluestore_scrub td/osd-scrub-repair 2026-02-17T11:34:04.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:483: TEST_auto_repair_bluestore_scrub: local dir=td/osd-scrub-repair 2026-02-17T11:34:04.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:484: TEST_auto_repair_bluestore_scrub: local poolname=testpool 2026-02-17T11:34:04.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:487: TEST_auto_repair_bluestore_scrub: run_mon td/osd-scrub-repair a 2026-02-17T11:34:04.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T11:34:04.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T11:34:04.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T11:34:04.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T11:34:04.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T11:34:04.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T11:34:04.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T11:34:04.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:34:04.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:34:04.923 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:34:04.923 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:34:04.923 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:34:04.924 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:34:04.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T11:34:04.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T11:34:04.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T11:34:04.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:34:04.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:34:04.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T11:34:04.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T11:34:04.953 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:34:04.953 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:34:04.953 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:34:04.953 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:34:04.953 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:34:04.954 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:34:04.954 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:34:04.954 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:34:04.954 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T11:34:05.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T11:34:05.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:34:05.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:34:05.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T11:34:05.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T11:34:05.012 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:34:05.012 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:34:05.012 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:34:05.012 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:34:05.012 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:34:05.012 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:34:05.013 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:34:05.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:34:05.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T11:34:05.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:488: TEST_auto_repair_bluestore_scrub: run_mgr td/osd-scrub-repair x 2026-02-17T11:34:05.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T11:34:05.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T11:34:05.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T11:34:05.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T11:34:05.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T11:34:05.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T11:34:05.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T11:34:05.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:34:05.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:34:05.158 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:34:05.159 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:34:05.159 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:34:05.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:34:05.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:34:05.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:34:05.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:492: TEST_auto_repair_bluestore_scrub: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-17T11:34:05.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:493: TEST_auto_repair_bluestore_scrub: seq 0 2 2026-02-17T11:34:05.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:493: TEST_auto_repair_bluestore_scrub: for id in $(seq 0 2) 2026-02-17T11:34:05.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:494: TEST_auto_repair_bluestore_scrub: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T11:34:05.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:34:05.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:34:05.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T11:34:05.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:34:05.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:34:05.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:34:05.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:34:05.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:34:05.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:34:05.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:34:05.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:34:05.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:34:05.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:34:05.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:34:05.183 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:34:05.183 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:34:05.183 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:34:05.183 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:34:05.183 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:34:05.184 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:34:05.184 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:34:05.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:34:05.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:34:05.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:34:05.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:34:05.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:34:05.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:34:05.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:34:05.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:34:05.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:34:05.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:34:05.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:34:05.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-17T11:34:05.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:34:05.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:34:05.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=64c8caf1-d4d4-422b-833e-1da0c7a95781 2026-02-17T11:34:05.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 64c8caf1-d4d4-422b-833e-1da0c7a95781' 2026-02-17T11:34:05.186 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 64c8caf1-d4d4-422b-833e-1da0c7a95781 2026-02-17T11:34:05.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:34:05.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAtUpRplMRkCxAAey8s9hoQGEMPblVyJk+qfA== 2026-02-17T11:34:05.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAtUpRplMRkCxAAey8s9hoQGEMPblVyJk+qfA=="}' 2026-02-17T11:34:05.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 64c8caf1-d4d4-422b-833e-1da0c7a95781 -i td/osd-scrub-repair/0/new.json 2026-02-17T11:34:05.287 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:34:05.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T11:34:05.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 --mkfs --key AQAtUpRplMRkCxAAey8s9hoQGEMPblVyJk+qfA== --osd-uuid 64c8caf1-d4d4-422b-833e-1da0c7a95781 2026-02-17T11:34:05.314 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:05.307+0000 7efc4113a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:05.315 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:05.307+0000 7efc4113a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:05.317 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:05.311+0000 7efc4113a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:05.317 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:05.311+0000 7efc4113a8c0 -1 bdev(0x55f85da01c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:34:05.317 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:05.311+0000 7efc4113a8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T11:34:08.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T11:34:08.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:34:08.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T11:34:08.098 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:34:08.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:34:08.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T11:34:08.289 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:34:08.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T11:34:08.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:34:08.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:34:08.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:34:08.312 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:08.306+0000 7f7dc6c7a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:08.314 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:08.306+0000 7f7dc6c7a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:08.315 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:08.306+0000 7f7dc6c7a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:08.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T11:34:08.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:34:08.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:34:08.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:34:08.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:34:08.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:08.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:34:08.445 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:34:08.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:08.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:34:08.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:09.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:09.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:09.595 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:34:09.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:34:09.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:09.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:34:09.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:09.786 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:09.778+0000 7f7dc6c7a8c0 -1 Falling back to public interface 2026-02-17T11:34:10.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:10.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:10.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:34:10.747 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:34:10.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:10.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:34:10.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:11.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:11.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:11.898 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:34:11.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:34:11.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:11.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:34:12.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:12.739 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:12.730+0000 7f7dc6c7a8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:34:13.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:13.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:13.058 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:34:13.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:34:13.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:13.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:34:13.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:14.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:14.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:14.209 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:34:14.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:34:14.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:14.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:34:14.366 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3624011595,v1:127.0.0.1:6803/3624011595] [v2:127.0.0.1:6804/3624011595,v1:127.0.0.1:6805/3624011595] exists,up 64c8caf1-d4d4-422b-833e-1da0c7a95781 2026-02-17T11:34:14.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:34:14.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:34:14.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:34:14.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:493: TEST_auto_repair_bluestore_scrub: for id in $(seq 0 2) 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:494: TEST_auto_repair_bluestore_scrub: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:34:14.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-17T11:34:14.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:34:14.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:34:14.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=33b7b071-a145-46dd-8f9b-64044f93a3e0 2026-02-17T11:34:14.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 33b7b071-a145-46dd-8f9b-64044f93a3e0' 2026-02-17T11:34:14.371 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 33b7b071-a145-46dd-8f9b-64044f93a3e0 2026-02-17T11:34:14.372 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:34:14.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA2UpRp3zumFhAAehIgwHAkBuA76TDHjstScg== 2026-02-17T11:34:14.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA2UpRp3zumFhAAehIgwHAkBuA76TDHjstScg=="}' 2026-02-17T11:34:14.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 33b7b071-a145-46dd-8f9b-64044f93a3e0 -i td/osd-scrub-repair/1/new.json 2026-02-17T11:34:14.536 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:34:14.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T11:34:14.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 --mkfs --key AQA2UpRp3zumFhAAehIgwHAkBuA76TDHjstScg== --osd-uuid 33b7b071-a145-46dd-8f9b-64044f93a3e0 2026-02-17T11:34:14.562 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:14.554+0000 7f32bb3d78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:14.564 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:14.554+0000 7f32bb3d78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:14.564 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:14.558+0000 7f32bb3d78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:14.565 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:14.558+0000 7f32bb3d78c0 -1 bdev(0x55f77f9bfc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:34:14.565 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:14.558+0000 7f32bb3d78c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T11:34:16.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T11:34:16.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:34:16.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T11:34:16.887 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:34:16.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:34:17.068 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:34:17.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T11:34:17.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T11:34:17.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:34:17.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:34:17.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:34:17.092 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:17.086+0000 7f5d2144a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:17.094 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:17.086+0000 7f5d2144a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:17.095 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:17.086+0000 7f5d2144a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:17.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T11:34:17.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:34:17.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:34:17.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:34:17.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:34:17.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:17.223 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:34:17.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:34:17.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:17.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:34:17.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:18.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:18.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:18.372 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:34:18.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:34:18.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:18.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:34:18.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:18.593 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:18.586+0000 7f5d2144a8c0 -1 Falling back to public interface 2026-02-17T11:34:19.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:19.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:19.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:34:19.521 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:34:19.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:19.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:34:19.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:20.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:20.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:20.671 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:34:20.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:34:20.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:20.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:34:20.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:21.571 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:21.562+0000 7f5d2144a8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:34:21.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:21.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:21.821 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:34:21.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:34:21.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:21.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:34:21.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:22.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:22.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:22.972 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:34:22.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:34:22.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:22.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:34:23.122 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1980916879,v1:127.0.0.1:6811/1980916879] [v2:127.0.0.1:6812/1980916879,v1:127.0.0.1:6813/1980916879] exists,up 33b7b071-a145-46dd-8f9b-64044f93a3e0 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:493: TEST_auto_repair_bluestore_scrub: for id in $(seq 0 2) 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:494: TEST_auto_repair_bluestore_scrub: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:34:23.123 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:34:23.124 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:34:23.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:34:23.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:34:23.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:34:23.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:34:23.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:34:23.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:34:23.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:34:23.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:34:23.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:34:23.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:34:23.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:34:23.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:34:23.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-17T11:34:23.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:34:23.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:34:23.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=10fa68ce-9210-4008-a9f1-5f1e11dd5c51 2026-02-17T11:34:23.127 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 10fa68ce-9210-4008-a9f1-5f1e11dd5c51 2026-02-17T11:34:23.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 10fa68ce-9210-4008-a9f1-5f1e11dd5c51' 2026-02-17T11:34:23.127 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:34:23.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA/UpRp874dCBAAfRapwMjTWSzwofUSbaYhKg== 2026-02-17T11:34:23.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA/UpRp874dCBAAfRapwMjTWSzwofUSbaYhKg=="}' 2026-02-17T11:34:23.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 10fa68ce-9210-4008-a9f1-5f1e11dd5c51 -i td/osd-scrub-repair/2/new.json 2026-02-17T11:34:23.295 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:34:23.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T11:34:23.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 --mkfs --key AQA/UpRp874dCBAAfRapwMjTWSzwofUSbaYhKg== --osd-uuid 10fa68ce-9210-4008-a9f1-5f1e11dd5c51 2026-02-17T11:34:23.322 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:23.314+0000 7f11258e68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:23.323 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:23.314+0000 7f11258e68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:23.324 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:23.318+0000 7f11258e68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:23.324 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:23.318+0000 7f11258e68c0 -1 bdev(0x55c8b3d8dc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T11:34:23.325 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:23.318+0000 7f11258e68c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T11:34:25.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T11:34:25.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:34:25.594 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T11:34:25.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T11:34:25.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:34:25.784 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T11:34:25.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T11:34:25.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T11:34:25.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:34:25.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:34:25.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:34:25.801 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:25.794+0000 7f821bfe88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:25.803 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:25.794+0000 7f821bfe88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:25.803 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:25.794+0000 7f821bfe88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:25.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T11:34:25.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:34:25.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:34:25.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:34:25.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:34:25.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:25.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:34:25.943 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:34:25.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:25.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:34:26.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:27.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:27.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:27.092 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:34:27.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:34:27.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:27.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:34:27.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:27.278 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:27.270+0000 7f821bfe88c0 -1 Falling back to public interface 2026-02-17T11:34:28.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:28.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:28.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:34:28.238 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:34:28.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:28.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:34:28.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:29.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:29.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:29.388 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:34:29.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:34:29.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:29.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:34:29.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:30.251 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:30.242+0000 7f821bfe88c0 -1 osd.2 0 log_to_monitors true 2026-02-17T11:34:30.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:30.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:30.538 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:34:30.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:34:30.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:30.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:34:30.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:31.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:31.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:31.688 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:34:31.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:34:31.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:31.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:34:31.834 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2220612339,v1:127.0.0.1:6819/2220612339] [v2:127.0.0.1:6820/2220612339,v1:127.0.0.1:6821/2220612339] exists,up 10fa68ce-9210-4008-a9f1-5f1e11dd5c51 2026-02-17T11:34:31.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:34:31.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:34:31.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:34:31.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:497: TEST_auto_repair_bluestore_scrub: create_pool testpool 1 1 2026-02-17T11:34:31.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create testpool 1 1 2026-02-17T11:34:31.979 INFO:tasks.workunit.client.0.trial128.stderr:pool 'testpool' created 2026-02-17T11:34:31.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:34:32.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:498: TEST_auto_repair_bluestore_scrub: ceph osd pool set testpool size 2 2026-02-17T11:34:33.180 INFO:tasks.workunit.client.0.trial128.stderr:set pool 1 size to 2 2026-02-17T11:34:33.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:499: TEST_auto_repair_bluestore_scrub: wait_for_clean 2026-02-17T11:34:33.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:34:33.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:34:33.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:34:33.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:34:33.193 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:34:33.193 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:34:33.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:34:33.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:34:33.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:34:33.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:34:33.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:34:33.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:34:33.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:34:33.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:34:33.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:34:33.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:34:33.457 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:34:33.457 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:34:33.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:34:33.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:34:33.457 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:34:33.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T11:34:33.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T11:34:33.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T11:34:33.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:34:33.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:34:33.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T11:34:33.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T11:34:33.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-17T11:34:33.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:34:33.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:34:33.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509442 2026-02-17T11:34:33.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509442 2026-02-17T11:34:33.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-64424509442' 2026-02-17T11:34:33.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:34:33.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T11:34:33.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:34:33.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:34:33.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T11:34:33.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:34:33.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T11:34:33.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T11:34:33.677 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836485 2026-02-17T11:34:33.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:34:33.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T11:34:33.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:34:34.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:34:34.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:34:34.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T11:34:34.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:34:35.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:34:35.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:34:36.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T11:34:36.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:34:36.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T11:34:36.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:34:36.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:34:36.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T11:34:36.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:34:36.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T11:34:36.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T11:34:36.140 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672964 2026-02-17T11:34:36.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:34:36.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T11:34:36.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:34:36.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509442 2026-02-17T11:34:36.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:34:36.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:34:36.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509442 2026-02-17T11:34:36.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:34:36.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509442 2026-02-17T11:34:36.309 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 64424509442 2026-02-17T11:34:36.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509442' 2026-02-17T11:34:36.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:34:36.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509442 -lt 64424509442 2026-02-17T11:34:36.460 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:34:36.460 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:34:36.460 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:34:36.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:34:36.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:34:36.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:34:36.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:34:36.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:34:36.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:34:36.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:34:36.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:34:36.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T11:34:36.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:34:36.793 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:34:36.793 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:34:36.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T11:34:36.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T11:34:36.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T11:34:36.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T11:34:36.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:34:36.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T11:34:37.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:34:37.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:34:37.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:34:37.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:34:37.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:34:37.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:34:37.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:34:37.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:34:37.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T11:34:37.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:34:37.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:34:37.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:34:37.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T11:34:37.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T11:34:37.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:34:37.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:34:37.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:34:37.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:34:37.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:34:37.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:34:37.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:34:37.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:34:37.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:34:37.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T11:34:37.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:34:37.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T11:34:37.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:34:37.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:34:37.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:34:37.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:34:37.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:34:37.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:34:37.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:34:37.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:34:37.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T11:34:37.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:34:37.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:34:37.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:34:38.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T11:34:38.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T11:34:38.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:34:38.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:34:38.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:34:38.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:34:38.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:34:38.146 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:34:38.146 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:34:38.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:34:38.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:34:38.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T11:34:38.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:34:38.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T11:34:38.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:34:38.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:34:38.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:34:38.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:34:38.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:34:38.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:34:38.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:34:38.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:34:38.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T11:34:38.888 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:34:38.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:34:38.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:34:39.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T11:34:39.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T11:34:39.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:34:39.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:34:39.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:34:39.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:34:39.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:34:39.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:34:39.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:34:39.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:34:39.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:34:39.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 13 )) 2026-02-17T11:34:39.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:34:39.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T11:34:40.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:34:40.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:34:40.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:34:40.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:34:40.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:34:40.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:34:40.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:34:40.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:34:40.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:34:40.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:34:40.214 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:34:40.214 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:34:40.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:34:40.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:34:40.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:34:40.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:502: TEST_auto_repair_bluestore_scrub: local payload=ABCDEF 2026-02-17T11:34:40.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:503: TEST_auto_repair_bluestore_scrub: echo ABCDEF 2026-02-17T11:34:40.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:504: TEST_auto_repair_bluestore_scrub: rados --pool testpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T11:34:40.422 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:508: TEST_auto_repair_bluestore_scrub: get_not_primary testpool SOMETHING 2026-02-17T11:34:40.422 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T11:34:40.422 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:34:40.422 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T11:34:40.423 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:34:40.423 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:34:40.423 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:34:40.423 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:34:40.574 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T11:34:40.574 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:34:40.574 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T11:34:40.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:508: TEST_auto_repair_bluestore_scrub: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T11:34:40.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:34:40.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:34:40.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:34:40.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:34:40.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T11:34:40.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:34:40.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:34:40.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:34:40.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:34:40.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:34:40.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:34:40.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:34:40.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:34:40.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:34:40.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:34:42.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:34:42.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T11:34:42.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:34:42.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:34:42.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:34:42.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:34:42.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:34:42.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-17T11:34:42.682 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-17T11:34:43.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T11:34:43.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:34:43.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:34:43.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:34:43.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:34:43.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:34:43.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:34:43.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:34:43.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:34:43.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:34:43.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:34:43.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:34:43.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:34:43.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:34:43.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:34:43.219 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:34:43.219 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:34:43.219 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:34:43.219 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:34:43.219 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:34:43.219 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:34:43.220 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:34:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:34:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:34:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:34:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:34:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:34:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:34:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:34:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:34:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:34:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-17T11:34:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:34:43.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:34:43.221 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:34:43.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T11:34:43.221 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:34:43.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:34:43.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:34:43.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:34:43.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:34:43.245 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:43.238+0000 7fc50afc38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:43.246 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:43.238+0000 7fc50afc38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:43.247 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:43.238+0000 7fc50afc38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:34:43.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:34:43.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:34:43.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:34:43.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:34:43.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:34:43.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:43.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:34:43.373 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:34:43.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:43.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:34:43.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:44.486 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:44.478+0000 7fc50afc38c0 -1 Falling back to public interface 2026-02-17T11:34:44.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:44.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:44.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:34:44.533 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:34:44.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:44.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:34:44.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:45.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:45.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:45.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:34:45.688 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:34:45.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:45.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:34:45.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:46.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:46.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:46.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:34:46.843 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:34:46.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:46.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:34:46.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:34:47.521 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:34:47.514+0000 7fc50afc38c0 -1 osd.0 20 log_to_monitors true 2026-02-17T11:34:47.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:34:47.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:34:47.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:34:47.998 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:34:47.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:34:47.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:34:48.152 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 24 up_thru 0 down_at 21 last_clean_interval [5,20) [v2:127.0.0.1:6802/1446673379,v1:127.0.0.1:6803/1446673379] [v2:127.0.0.1:6804/1446673379,v1:127.0.0.1:6805/1446673379] exists,up 64c8caf1-d4d4-422b-833e-1da0c7a95781 2026-02-17T11:34:48.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:34:48.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:34:48.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:34:48.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:34:48.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:34:48.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:34:48.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:34:48.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:34:48.153 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:34:48.153 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:34:48.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:34:48.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:34:48.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:34:48.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:34:48.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:34:48.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:34:48.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:34:48.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:34:48.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:34:48.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:34:48.409 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:34:48.409 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:34:48.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:34:48.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:34:48.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:34:48.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215106 2026-02-17T11:34:48.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215106 2026-02-17T11:34:48.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215106' 2026-02-17T11:34:48.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:34:48.483 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:34:48.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672968 2026-02-17T11:34:48.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672968 2026-02-17T11:34:48.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215106 1-42949672968' 2026-02-17T11:34:48.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:34:48.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:34:48.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509446 2026-02-17T11:34:48.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509446 2026-02-17T11:34:48.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215106 1-42949672968 2-64424509446' 2026-02-17T11:34:48.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:34:48.627 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-103079215106 2026-02-17T11:34:48.627 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:34:48.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:34:48.629 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-103079215106 2026-02-17T11:34:48.629 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:34:48.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215106 2026-02-17T11:34:48.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 103079215106' 2026-02-17T11:34:48.631 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 103079215106 2026-02-17T11:34:48.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:34:48.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 103079215106 2026-02-17T11:34:48.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:34:49.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:34:49.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:34:49.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215106 -lt 103079215106 2026-02-17T11:34:49.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:34:49.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672968 2026-02-17T11:34:49.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:34:49.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:34:49.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672968 2026-02-17T11:34:49.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:34:49.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672968 2026-02-17T11:34:49.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2026-02-17T11:34:49.951 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 42949672968 2026-02-17T11:34:49.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:34:50.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672968 -lt 42949672968 2026-02-17T11:34:50.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:34:50.113 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509446 2026-02-17T11:34:50.113 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:34:50.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:34:50.114 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509446 2026-02-17T11:34:50.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:34:50.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509446 2026-02-17T11:34:50.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509446' 2026-02-17T11:34:50.116 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 64424509446 2026-02-17T11:34:50.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:34:50.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509446 -lt 64424509446 2026-02-17T11:34:50.269 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:34:50.269 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:34:50.269 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:34:50.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:34:50.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:34:50.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:34:50.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:34:50.448 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:34:50.448 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:34:50.448 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:34:50.448 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:34:50.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:34:50.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:34:50.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:34:50.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:34:50.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:34:50.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:34:50.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:34:50.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:510: TEST_auto_repair_bluestore_scrub: get_pg testpool SOMETHING 2026-02-17T11:34:50.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=testpool 2026-02-17T11:34:50.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:34:50.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map testpool SOMETHING 2026-02-17T11:34:50.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:34:50.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:510: TEST_auto_repair_bluestore_scrub: local pgid=1.0 2026-02-17T11:34:50.929 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:511: TEST_auto_repair_bluestore_scrub: get_primary testpool SOMETHING 2026-02-17T11:34:50.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:34:50.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:34:50.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:34:50.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:34:51.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:511: TEST_auto_repair_bluestore_scrub: local primary=1 2026-02-17T11:34:51.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:512: TEST_auto_repair_bluestore_scrub: get_last_scrub_stamp 1.0 2026-02-17T11:34:51.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:34:51.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:34:51.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:34:51.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:34:51.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:512: TEST_auto_repair_bluestore_scrub: local last_scrub_stamp=2026-02-17T11:34:31.974929+0000 2026-02-17T11:34:51.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:513: TEST_auto_repair_bluestore_scrub: ceph tell 1.0 schedule-scrub 2026-02-17T11:34:51.289 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T11:34:51.289 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T11:34:51.289 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T11:34:51.289 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-10T11:33:11.286560+0000" 2026-02-17T11:34:51.289 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:34:51.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:516: TEST_auto_repair_bluestore_scrub: wait_for_scrub 1.0 2026-02-17T11:34:31.974929+0000 2026-02-17T11:34:51.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T11:34:51.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:34:31.974929+0000 2026-02-17T11:34:51.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:34:51.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:34:51.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:34:51.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:34:51.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:34:51.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:34:51.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:34:51.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:34:51.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:34:31.974929+0000 '>' 2026-02-17T11:34:31.974929+0000 2026-02-17T11:34:51.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:34:52.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:34:52.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:34:52.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:34:52.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:34:52.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:34:52.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:34:52.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:34:52.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:34:31.974929+0000 '>' 2026-02-17T11:34:31.974929+0000 2026-02-17T11:34:52.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:34:53.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:34:53.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:34:53.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T11:34:53.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:34:53.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:34:53.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:34:53.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:34:53.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:34:51.757153+0000 '>' 2026-02-17T11:34:31.974929+0000 2026-02-17T11:34:53.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:34:53.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:517: TEST_auto_repair_bluestore_scrub: ceph pg dump pgs 2026-02-17T11:34:53.896 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T11:34:53.896 INFO:tasks.workunit.client.0.trial128.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean+inconsistent 2026-02-17T11:34:51.757213+0000 20'1 25:51 [1,0] 1 [1,0] 1 20'1 2026-02-17T11:34:51.757153+0000 0'0 2026-02-17T11:34:31.974929+0000 0 0 queued for deep scrub 1 0 2026-02-17T11:34:53.896 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:34:53.897 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T11:34:53.897 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:34:53.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:519: TEST_auto_repair_bluestore_scrub: sleep 2 2026-02-17T11:34:55.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:520: TEST_auto_repair_bluestore_scrub: ceph pg dump pgs 2026-02-17T11:34:56.051 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T11:34:56.051 INFO:tasks.workunit.client.0.trial128.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean+inconsistent 2026-02-17T11:34:51.757213+0000 20'1 25:51 [1,0] 1 [1,0] 1 20'1 2026-02-17T11:34:51.757153+0000 0'0 2026-02-17T11:34:31.974929+0000 0 0 queued for deep scrub 1 0 2026-02-17T11:34:56.051 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:34:56.051 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T11:34:56.052 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:34:56.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:521: TEST_auto_repair_bluestore_scrub: sleep 2 2026-02-17T11:34:58.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:522: TEST_auto_repair_bluestore_scrub: ceph pg dump pgs 2026-02-17T11:34:58.204 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T11:34:58.204 INFO:tasks.workunit.client.0.trial128.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean+inconsistent 2026-02-17T11:34:51.757213+0000 20'1 25:51 [1,0] 1 [1,0] 1 20'1 2026-02-17T11:34:51.757153+0000 0'0 2026-02-17T11:34:31.974929+0000 0 0 queued for deep scrub 1 0 2026-02-17T11:34:58.204 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:34:58.204 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T11:34:58.205 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:34:58.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:523: TEST_auto_repair_bluestore_scrub: sleep 5 2026-02-17T11:35:03.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:524: TEST_auto_repair_bluestore_scrub: wait_for_clean 2026-02-17T11:35:03.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:35:03.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:35:03.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:35:03.215 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:35:03.215 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:35:03.216 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:35:03.216 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:35:03.216 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:35:03.216 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:35:03.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:35:03.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:35:03.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:35:03.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:35:03.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:35:03.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:35:03.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:35:03.485 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:35:03.485 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:35:03.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:35:03.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:35:03.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:35:03.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=103079215110 2026-02-17T11:35:03.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 103079215110 2026-02-17T11:35:03.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215110' 2026-02-17T11:35:03.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:35:03.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:35:03.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672972 2026-02-17T11:35:03.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672972 2026-02-17T11:35:03.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215110 1-42949672972' 2026-02-17T11:35:03.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:35:03.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:35:03.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509450 2026-02-17T11:35:03.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509450 2026-02-17T11:35:03.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-103079215110 1-42949672972 2-64424509450' 2026-02-17T11:35:03.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:35:03.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-103079215110 2026-02-17T11:35:03.705 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:35:03.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:35:03.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-103079215110 2026-02-17T11:35:03.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:35:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=103079215110 2026-02-17T11:35:03.708 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 103079215110 2026-02-17T11:35:03.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 103079215110' 2026-02-17T11:35:03.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:35:03.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215109 -lt 103079215110 2026-02-17T11:35:03.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:35:04.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:35:04.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:35:05.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215109 -lt 103079215110 2026-02-17T11:35:05.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:35:06.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:35:06.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:35:06.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215110 -lt 103079215110 2026-02-17T11:35:06.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:35:06.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672972 2026-02-17T11:35:06.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:35:06.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:35:06.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672972 2026-02-17T11:35:06.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:35:06.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672972 2026-02-17T11:35:06.177 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672972 2026-02-17T11:35:06.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672972' 2026-02-17T11:35:06.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:35:06.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672972 -lt 42949672972 2026-02-17T11:35:06.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:35:06.333 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509450 2026-02-17T11:35:06.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:35:06.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:35:06.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509450 2026-02-17T11:35:06.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:35:06.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509450 2026-02-17T11:35:06.337 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 64424509450 2026-02-17T11:35:06.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509450' 2026-02-17T11:35:06.338 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:35:06.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509450 -lt 64424509450 2026-02-17T11:35:06.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:35:06.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:35:06.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:35:06.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:35:06.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:35:06.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:35:06.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:35:06.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:35:06.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:35:06.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:35:06.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:35:06.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:35:06.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:35:06.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:35:06.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:35:06.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:35:06.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:35:06.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:35:06.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:525: TEST_auto_repair_bluestore_scrub: ceph pg dump pgs 2026-02-17T11:35:07.132 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T11:35:07.132 INFO:tasks.workunit.client.0.trial128.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean 2026-02-17T11:34:52.740911+0000 20'1 25:66 [1,0] 1 [1,0] 1 20'1 2026-02-17T11:34:52.738703+0000 20'1 2026-02-17T11:34:52.738703+0000 0 0 periodic scrub scheduled @ 2026-02-18T11:34:52.738703+0000 1 0 2026-02-17T11:35:07.132 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:35:07.132 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T11:35:07.133 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:35:07.141 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:528: TEST_auto_repair_bluestore_scrub: get_not_primary testpool SOMETHING 2026-02-17T11:35:07.141 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T11:35:07.141 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:35:07.142 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T11:35:07.142 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:35:07.142 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:35:07.143 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:35:07.143 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:35:07.295 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T11:35:07.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:35:07.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T11:35:07.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:528: TEST_auto_repair_bluestore_scrub: objectstore_tool td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T11:35:07.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:35:07.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:35:07.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:35:07.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:35:07.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T11:35:07.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:35:07.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:35:07.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:35:07.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:35:07.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:35:07.456 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:35:07.456 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:35:07.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:35:07.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:35:07.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:35:08.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:35:08.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T11:35:08.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:35:08.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:35:08.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:35:08.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:35:08.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:35:08.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING list-attrs 2026-02-17T11:35:09.073 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:35:09.073 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:35:09.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:35:09.610 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:35:09.610 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:35:09.610 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:35:09.611 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:35:09.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:35:09.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:35:09.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:35:09.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:35:09.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:35:09.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:35:09.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:35:09.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:35:09.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:35:09.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0' 2026-02-17T11:35:09.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:35:09.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:35:09.612 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:35:09.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-scrub-backoff-ratio=0 2026-02-17T11:35:09.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:35:09.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:35:09.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:35:09.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:35:09.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:35:09.628 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:09.625+0000 7f7f6e2788c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:09.630 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:09.625+0000 7f7f6e2788c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:09.631 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:09.625+0000 7f7f6e2788c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:09.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:35:09.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:35:09.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:35:09.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:35:09.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:35:09.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:09.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:35:09.772 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:35:09.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:09.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:35:09.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:10.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:10.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:10.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:35:10.923 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:35:10.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:10.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:35:11.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:11.110 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:11.105+0000 7f7f6e2788c0 -1 Falling back to public interface 2026-02-17T11:35:12.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:12.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:12.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:35:12.074 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:35:12.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:12.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:35:12.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:13.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:13.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:13.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:35:13.225 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:35:13.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:13.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:35:13.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:14.067 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:14.061+0000 7f7f6e2788c0 -1 osd.0 25 log_to_monitors true 2026-02-17T11:35:14.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:14.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:14.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:35:14.374 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:35:14.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:14.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:35:14.521 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 29 up_thru 0 down_at 26 last_clean_interval [24,25) [v2:127.0.0.1:6802/4130320630,v1:127.0.0.1:6803/4130320630] [v2:127.0.0.1:6804/4130320630,v1:127.0.0.1:6805/4130320630] exists,up 64c8caf1-d4d4-422b-833e-1da0c7a95781 2026-02-17T11:35:14.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:35:14.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:35:14.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:35:14.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:35:14.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:35:14.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:35:14.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:35:14.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:35:14.522 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:35:14.522 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:35:14.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:35:14.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:35:14.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:35:14.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:35:14.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:35:14.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:35:14.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:35:14.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:35:14.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:35:14.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:35:14.784 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:35:14.785 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:35:14.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:35:14.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:35:14.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:35:14.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051586 2026-02-17T11:35:14.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051586 2026-02-17T11:35:14.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586' 2026-02-17T11:35:14.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:35:14.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:35:14.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672975 2026-02-17T11:35:14.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672975 2026-02-17T11:35:14.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586 1-42949672975' 2026-02-17T11:35:14.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:35:14.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:35:14.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509453 2026-02-17T11:35:14.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509453 2026-02-17T11:35:14.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586 1-42949672975 2-64424509453' 2026-02-17T11:35:14.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:35:15.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-124554051586 2026-02-17T11:35:15.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:35:15.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:35:15.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-124554051586 2026-02-17T11:35:15.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:35:15.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051586 2026-02-17T11:35:15.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 124554051586' 2026-02-17T11:35:15.003 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 124554051586 2026-02-17T11:35:15.004 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:35:15.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 103079215110 -lt 124554051586 2026-02-17T11:35:15.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:35:16.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:35:16.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:35:16.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051586 -lt 124554051586 2026-02-17T11:35:16.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:35:16.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672975 2026-02-17T11:35:16.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:35:16.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:35:16.319 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672975 2026-02-17T11:35:16.319 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:35:16.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672975 2026-02-17T11:35:16.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672975' 2026-02-17T11:35:16.321 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 42949672975 2026-02-17T11:35:16.321 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:35:16.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672975 -lt 42949672975 2026-02-17T11:35:16.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:35:16.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509453 2026-02-17T11:35:16.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:35:16.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:35:16.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509453 2026-02-17T11:35:16.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:35:16.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509453 2026-02-17T11:35:16.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509453' 2026-02-17T11:35:16.480 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 64424509453 2026-02-17T11:35:16.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:35:16.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509453 -lt 64424509453 2026-02-17T11:35:16.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:35:16.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:35:16.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:35:16.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:35:16.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:35:16.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:35:16.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:35:16.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:35:16.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:35:16.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:35:16.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:35:16.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:35:16.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:35:16.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:35:16.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:35:17.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:35:17.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:35:17.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:35:17.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:529: TEST_auto_repair_bluestore_scrub: rados --pool testpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:35:17.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:530: TEST_auto_repair_bluestore_scrub: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:35:17.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:531: TEST_auto_repair_bluestore_scrub: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-17T11:35:17.189 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:34:51.753+0000 7f5cff3c8640 10 osd.1 pg_epoch: 25 pg[1.0( v 20'1 (0'0,20'1] local-lis/les=24/25 n=1 ec=16/16 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0] r=0 lpr=24 crt=20'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing [ 1.0: DEEP_SCRUB_ON_ERROR ] planned DEEP_SCRUB_ON_ERROR] scrubber: scrub_finish before flags: DEEP_SCRUB_ON_ERROR. repair state: no-repair. deep_scrub_on_error: 1 2026-02-17T11:35:17.189 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:34:51.753+0000 7f5cff3c8640 15 osd.1 pg_epoch: 25 pg[1.0( v 20'1 (0'0,20'1] local-lis/les=24/25 n=1 ec=16/16 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0] r=0 lpr=24 crt=20'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing [ 1.0: DEEP_SCRUB_ON_ERROR ] ] scrubber: scrub_finish Try to auto repair after scrub errors 2026-02-17T11:35:17.189 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:34:51.753+0000 7f5cff3c8640 10 osd.1 pg_epoch: 25 pg[1.0( v 20'1 (0'0,20'1] local-lis/les=24/25 n=1 ec=16/16 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0] r=0 lpr=24 crt=20'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing [ 1.0: ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 0 2026-02-17T11:35:17.189 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:34:52.733+0000 7f5d033d0640 10 osd.1 pg_epoch: 25 pg[1.0( v 20'1 (0'0,20'1] local-lis/les=24/25 n=1 ec=16/16 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0] r=0 lpr=24 crt=20'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: AUTO_REPAIR ] planned AUTO_REPAIR TIME_FOR_DEEP] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T11:35:17.189 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:34:52.733+0000 7f5d033d0640 10 osd.1 pg_epoch: 25 pg[1.0( v 20'1 (0'0,20'1] local-lis/les=24/25 n=1 ec=16/16 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0] r=0 lpr=24 crt=20'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep+inconsistent [ 1.0: AUTO_REPAIR ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-17T11:35:17.189 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:34:52.733+0000 7f5d033d0640 15 osd.1 pg_epoch: 25 pg[1.0( v 20'1 (0'0,20'1] local-lis/les=24/25 n=1 ec=16/16 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0] r=0 lpr=24 crt=20'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+inconsistent+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish: 1 errors. 1 errors fixed 2026-02-17T11:35:17.189 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:34:52.733+0000 7f5d033d0640 20 osd.1 pg_epoch: 25 pg[1.0( v 20'1 (0'0,20'1] local-lis/les=24/25 n=1 ec=16/16 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0] r=0 lpr=24 crt=20'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+inconsistent+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish All may be fixed 2026-02-17T11:35:17.189 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:34:52.733+0000 7f5d033d0640 19 osd.1 pg_epoch: 25 pg[1.0( v 20'1 (0'0,20'1] local-lis/les=24/25 n=1 ec=16/16 lis/c=24/24 les/c/f=25/25/0 sis=24) [1,0] r=0 lpr=24 crt=20'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+inconsistent+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T11:35:17.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:534: TEST_auto_repair_bluestore_scrub: ceph pg 1.0 query 2026-02-17T11:35:17.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:534: TEST_auto_repair_bluestore_scrub: jq .info.stats.stat_sum.num_objects_repaired 2026-02-17T11:35:17.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:534: TEST_auto_repair_bluestore_scrub: COUNT=1 2026-02-17T11:35:17.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:535: TEST_auto_repair_bluestore_scrub: test 1 = 1 2026-02-17T11:35:17.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T11:35:17.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:35:17.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:35:17.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:35:17.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:35:17.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:35:17.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:35:17.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:35:17.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:35:17.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:35:17.373 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:35:17.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:35:17.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:35:17.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:35:17.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:35:17.377 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:35:17.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:35:17.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:35:17.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:35:17.379 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:35:17.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:35:17.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:35:17.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:35:17.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:35:17.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:35:17.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:35:17.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:35:17.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:35:17.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:35:17.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T11:35:17.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T11:35:17.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T11:35:17.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T11:35:17.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:35:17.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:35:17.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:35:17.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:35:17.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:35:17.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:35:17.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:35:17.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:35:17.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:35:17.397 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:35:17.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:35:17.398 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:35:17.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:35:17.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:35:17.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:35:17.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:35:17.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:35:17.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:35:17.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:35:17.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:35:17.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:35:17.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:35:17.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:35:17.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:35:17.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:35:17.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:35:17.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:35:17.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:35:17.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T11:35:17.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:35:17.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:35:17.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:35:17.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T11:35:17.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:35:17.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T11:35:17.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:35:17.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T11:35:17.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_bluestore_tag td/osd-scrub-repair 2026-02-17T11:35:17.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:383: TEST_auto_repair_bluestore_tag: local dir=td/osd-scrub-repair 2026-02-17T11:35:17.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:384: TEST_auto_repair_bluestore_tag: local poolname=testpool 2026-02-17T11:35:17.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:387: TEST_auto_repair_bluestore_tag: run_mon td/osd-scrub-repair a 2026-02-17T11:35:17.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T11:35:17.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T11:35:17.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T11:35:17.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T11:35:17.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T11:35:17.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T11:35:17.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T11:35:17.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:35:17.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:35:17.433 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:35:17.433 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:35:17.433 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:35:17.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:35:17.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T11:35:17.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T11:35:17.464 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T11:35:17.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:35:17.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:35:17.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T11:35:17.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T11:35:17.465 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:35:17.465 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:35:17.466 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:35:17.466 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:35:17.466 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:35:17.466 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:35:17.466 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:35:17.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:35:17.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T11:35:17.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T11:35:17.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:35:17.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:35:17.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T11:35:17.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T11:35:17.525 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:35:17.526 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:35:17.526 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:35:17.527 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:35:17.527 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:35:17.527 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:35:17.527 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:35:17.527 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:35:17.527 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T11:35:17.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:388: TEST_auto_repair_bluestore_tag: run_mgr td/osd-scrub-repair x 2026-02-17T11:35:17.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T11:35:17.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T11:35:17.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T11:35:17.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T11:35:17.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T11:35:17.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T11:35:17.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T11:35:17.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:35:17.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:35:17.678 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:35:17.679 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:35:17.679 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:35:17.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:35:17.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:35:17.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:35:17.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:395: TEST_auto_repair_bluestore_tag: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-17T11:35:17.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:396: TEST_auto_repair_bluestore_tag: seq 0 2 2026-02-17T11:35:17.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:396: TEST_auto_repair_bluestore_tag: for id in $(seq 0 2) 2026-02-17T11:35:17.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:397: TEST_auto_repair_bluestore_tag: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T11:35:17.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:35:17.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:35:17.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T11:35:17.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:35:17.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:35:17.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:35:17.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:35:17.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:35:17.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:35:17.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:35:17.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:35:17.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:35:17.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:35:17.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:35:17.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:35:17.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:35:17.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:35:17.702 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:35:17.702 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:35:17.702 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:35:17.703 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:35:17.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:35:17.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:35:17.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:35:17.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:35:17.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:35:17.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:35:17.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:35:17.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:35:17.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:35:17.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:35:17.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:35:17.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-17T11:35:17.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:35:17.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:35:17.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=0f717407-e69b-4661-baba-016b4f6edfe2 2026-02-17T11:35:17.704 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 0f717407-e69b-4661-baba-016b4f6edfe2 2026-02-17T11:35:17.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 0f717407-e69b-4661-baba-016b4f6edfe2' 2026-02-17T11:35:17.705 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:35:17.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB1UpRp7EJtKhAADPTQwZWjl4wgULXxfYhlsw== 2026-02-17T11:35:17.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB1UpRp7EJtKhAADPTQwZWjl4wgULXxfYhlsw=="}' 2026-02-17T11:35:17.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 0f717407-e69b-4661-baba-016b4f6edfe2 -i td/osd-scrub-repair/0/new.json 2026-02-17T11:35:17.804 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:35:17.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T11:35:17.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq --mkfs --key AQB1UpRp7EJtKhAADPTQwZWjl4wgULXxfYhlsw== --osd-uuid 0f717407-e69b-4661-baba-016b4f6edfe2 2026-02-17T11:35:17.830 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:17.825+0000 7f22672e28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:17.831 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:17.825+0000 7f22672e28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:17.832 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:17.829+0000 7f22672e28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:17.833 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:17.829+0000 7f22672e28c0 -1 bdev(0x560956cdfc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:35:17.833 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:17.829+0000 7f22672e28c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T11:35:20.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T11:35:20.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:35:20.727 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:35:20.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T11:35:20.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:35:20.925 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:35:20.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T11:35:20.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T11:35:20.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:35:20.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:35:20.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:35:20.950 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:20.945+0000 7faa6e05d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:20.952 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:20.945+0000 7faa6e05d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:20.953 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:20.949+0000 7faa6e05d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:21.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T11:35:21.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:35:21.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:35:21.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:35:21.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:35:21.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:21.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:35:21.078 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:35:21.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:21.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:35:21.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:22.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:22.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:22.233 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:35:22.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:35:22.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:22.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:35:22.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:22.442 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:22.437+0000 7faa6e05d8c0 -1 Falling back to public interface 2026-02-17T11:35:23.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:23.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:23.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:35:23.390 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:35:23.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:23.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:35:23.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:24.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:24.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:24.545 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:35:24.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:35:24.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:24.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:35:24.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:25.672 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:25.665+0000 7faa6e05d8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:35:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:35:25.700 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:35:25.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:25.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:35:25.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:26.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:26.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:26.856 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:35:26.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:35:26.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:26.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:35:27.007 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2973240441,v1:127.0.0.1:6803/2973240441] [v2:127.0.0.1:6804/2973240441,v1:127.0.0.1:6805/2973240441] exists,up 0f717407-e69b-4661-baba-016b4f6edfe2 2026-02-17T11:35:27.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:35:27.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:35:27.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:35:27.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:396: TEST_auto_repair_bluestore_tag: for id in $(seq 0 2) 2026-02-17T11:35:27.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:397: TEST_auto_repair_bluestore_tag: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T11:35:27.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:35:27.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:35:27.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:35:27.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:35:27.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:35:27.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:35:27.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:35:27.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:35:27.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:35:27.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:35:27.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:35:27.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:35:27.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:35:27.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:35:27.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:35:27.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-17T11:35:27.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:35:27.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:35:27.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=6a74c5f2-6ec9-4c86-896b-b8d55e2f34ad 2026-02-17T11:35:27.013 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 6a74c5f2-6ec9-4c86-896b-b8d55e2f34ad 2026-02-17T11:35:27.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 6a74c5f2-6ec9-4c86-896b-b8d55e2f34ad' 2026-02-17T11:35:27.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:35:27.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB/UpRpwt9OARAALSJ8H4fQNhOziKgjum/Bsg== 2026-02-17T11:35:27.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB/UpRpwt9OARAALSJ8H4fQNhOziKgjum/Bsg=="}' 2026-02-17T11:35:27.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 6a74c5f2-6ec9-4c86-896b-b8d55e2f34ad -i td/osd-scrub-repair/1/new.json 2026-02-17T11:35:27.182 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:35:27.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T11:35:27.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq --mkfs --key AQB/UpRpwt9OARAALSJ8H4fQNhOziKgjum/Bsg== --osd-uuid 6a74c5f2-6ec9-4c86-896b-b8d55e2f34ad 2026-02-17T11:35:27.210 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:27.205+0000 7f62653d98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:27.211 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:27.205+0000 7f62653d98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:27.212 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:27.209+0000 7f62653d98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:27.213 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:27.209+0000 7f62653d98c0 -1 bdev(0x55cc97945c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:35:27.213 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:27.209+0000 7f62653d98c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T11:35:29.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T11:35:29.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:35:29.735 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:35:29.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T11:35:29.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:35:29.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T11:35:29.932 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:35:29.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T11:35:29.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:35:29.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:35:29.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:35:29.956 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:29.949+0000 7f9a6c2048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:29.958 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:29.953+0000 7f9a6c2048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:29.959 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:29.953+0000 7f9a6c2048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:30.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T11:35:30.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:35:30.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:35:30.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:35:30.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:35:30.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:30.082 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:35:30.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:35:30.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:30.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:35:30.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:31.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:31.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:31.239 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:35:31.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:35:31.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:31.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:35:31.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:31.662 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:31.657+0000 7f9a6c2048c0 -1 Falling back to public interface 2026-02-17T11:35:32.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:32.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:32.392 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:35:32.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:35:32.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:32.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:35:32.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:33.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:33.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:33.546 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:35:33.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:35:33.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:33.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:35:33.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:34.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:34.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:34.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:35:34.705 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:35:34.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:34.706 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:34.692+0000 7f9a6c2048c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:35:34.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:35:34.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:35.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:35.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:35.855 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:35:35.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:35:35.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:35.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:35:36.007 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/472801850,v1:127.0.0.1:6811/472801850] [v2:127.0.0.1:6812/472801850,v1:127.0.0.1:6813/472801850] exists,up 6a74c5f2-6ec9-4c86-896b-b8d55e2f34ad 2026-02-17T11:35:36.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:35:36.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:35:36.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:35:36.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:396: TEST_auto_repair_bluestore_tag: for id in $(seq 0 2) 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:397: TEST_auto_repair_bluestore_tag: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:35:36.009 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:35:36.010 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:35:36.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:35:36.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:35:36.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:35:36.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:35:36.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:35:36.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:35:36.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:35:36.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:35:36.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:35:36.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:35:36.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:35:36.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:35:36.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-17T11:35:36.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:35:36.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:35:36.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=fd509e57-d03d-4903-b0e4-3beb810f9cad 2026-02-17T11:35:36.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 fd509e57-d03d-4903-b0e4-3beb810f9cad' 2026-02-17T11:35:36.012 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 fd509e57-d03d-4903-b0e4-3beb810f9cad 2026-02-17T11:35:36.012 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:35:36.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCIUpRpi9UwARAASbWx+UsUZ8z20MYi6nAlDA== 2026-02-17T11:35:36.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCIUpRpi9UwARAASbWx+UsUZ8z20MYi6nAlDA=="}' 2026-02-17T11:35:36.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new fd509e57-d03d-4903-b0e4-3beb810f9cad -i td/osd-scrub-repair/2/new.json 2026-02-17T11:35:36.183 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:35:36.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T11:35:36.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq --mkfs --key AQCIUpRpi9UwARAASbWx+UsUZ8z20MYi6nAlDA== --osd-uuid fd509e57-d03d-4903-b0e4-3beb810f9cad 2026-02-17T11:35:36.210 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:36.204+0000 7f89add3f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:36.211 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:36.204+0000 7f89add3f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:36.212 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:36.208+0000 7f89add3f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:36.213 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:36.208+0000 7f89add3f8c0 -1 bdev(0x55b1aa503c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T11:35:36.213 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:36.208+0000 7f89add3f8c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T11:35:38.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T11:35:38.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:35:38.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T11:35:38.490 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T11:35:38.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:35:38.688 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T11:35:38.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T11:35:38.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T11:35:38.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:35:38.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:35:38.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:35:38.714 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:38.708+0000 7fdfc358d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:38.716 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:38.708+0000 7fdfc358d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:38.717 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:38.712+0000 7fdfc358d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:38.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T11:35:38.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:35:38.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:35:38.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:35:38.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:35:38.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:38.838 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:35:38.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:35:38.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:38.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:35:38.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:39.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:39.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:39.989 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:35:39.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:35:39.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:39.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:35:40.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:40.190 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:40.184+0000 7fdfc358d8c0 -1 Falling back to public interface 2026-02-17T11:35:41.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:41.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:41.143 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:35:41.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:35:41.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:41.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:35:41.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:42.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:42.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:42.301 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:35:42.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:35:42.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:42.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:35:42.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:43.217 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:43.212+0000 7fdfc358d8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T11:35:43.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:43.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:43.454 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:35:43.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:35:43.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:43.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:35:43.606 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3978336816,v1:127.0.0.1:6819/3978336816] [v2:127.0.0.1:6820/3978336816,v1:127.0.0.1:6821/3978336816] exists,up fd509e57-d03d-4903-b0e4-3beb810f9cad 2026-02-17T11:35:43.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:35:43.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:35:43.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:35:43.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:400: TEST_auto_repair_bluestore_tag: create_pool testpool 1 1 2026-02-17T11:35:43.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create testpool 1 1 2026-02-17T11:35:43.779 INFO:tasks.workunit.client.0.trial128.stderr:pool 'testpool' created 2026-02-17T11:35:43.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:35:44.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:401: TEST_auto_repair_bluestore_tag: ceph osd pool set testpool size 2 2026-02-17T11:35:44.989 INFO:tasks.workunit.client.0.trial128.stderr:set pool 1 size to 2 2026-02-17T11:35:44.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:402: TEST_auto_repair_bluestore_tag: wait_for_clean 2026-02-17T11:35:45.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:35:45.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:35:45.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:35:45.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:35:45.001 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:35:45.001 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:35:45.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:35:45.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:35:45.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:35:45.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:35:45.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:35:45.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:35:45.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:35:45.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:35:45.119 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:35:45.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:35:45.272 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:35:45.272 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:35:45.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:35:45.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:35:45.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:35:45.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T11:35:45.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T11:35:45.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T11:35:45.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:35:45.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:35:45.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705667 2026-02-17T11:35:45.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705667 2026-02-17T11:35:45.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705667' 2026-02-17T11:35:45.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:35:45.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:35:45.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574850 2026-02-17T11:35:45.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574850 2026-02-17T11:35:45.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705667 2-55834574850' 2026-02-17T11:35:45.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:35:45.487 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T11:35:45.487 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:35:45.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:35:45.489 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T11:35:45.489 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:35:45.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T11:35:45.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T11:35:45.491 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836485 2026-02-17T11:35:45.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:35:45.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T11:35:45.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:35:46.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:35:46.655 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:35:46.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-17T11:35:46.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:35:46.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705667 2026-02-17T11:35:46.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:35:46.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:35:46.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705667 2026-02-17T11:35:46.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:35:46.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705667 2026-02-17T11:35:46.810 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705667 2026-02-17T11:35:46.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-17T11:35:46.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:35:46.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705668 -lt 38654705667 2026-02-17T11:35:46.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:35:46.967 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574850 2026-02-17T11:35:46.967 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:35:46.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:35:46.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574850 2026-02-17T11:35:46.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:35:46.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574850 2026-02-17T11:35:46.969 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574850 2026-02-17T11:35:46.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574850' 2026-02-17T11:35:46.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:35:47.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574850 -lt 55834574850 2026-02-17T11:35:47.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:35:47.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:35:47.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:35:47.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:35:47.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:35:47.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:35:47.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:35:47.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:35:47.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:35:47.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:35:47.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:35:47.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:35:47.456 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:35:47.456 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:35:47.456 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:35:47.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:35:47.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:35:47.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:35:47.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:405: TEST_auto_repair_bluestore_tag: local payload=ABCDEF 2026-02-17T11:35:47.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:406: TEST_auto_repair_bluestore_tag: echo ABCDEF 2026-02-17T11:35:47.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:407: TEST_auto_repair_bluestore_tag: rados --pool testpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T11:35:47.659 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:411: TEST_auto_repair_bluestore_tag: get_not_primary testpool SOMETHING 2026-02-17T11:35:47.659 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T11:35:47.659 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:35:47.660 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T11:35:47.660 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:35:47.660 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:35:47.660 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:35:47.660 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:35:47.812 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T11:35:47.812 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:35:47.813 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T11:35:47.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:411: TEST_auto_repair_bluestore_tag: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T11:35:47.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:35:47.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:35:47.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:35:47.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:35:47.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T11:35:47.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:35:47.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:35:47.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:35:47.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:35:47.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:35:47.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:35:47.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:35:47.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:35:47.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:35:47.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:35:49.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:35:49.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T11:35:49.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:35:49.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:35:49.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:35:49.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:35:49.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:35:49.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-17T11:35:49.937 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-17T11:35:50.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T11:35:50.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:35:50.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:35:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:35:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:35:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:35:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:35:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:35:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:35:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:35:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:35:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:35:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:35:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:35:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:35:50.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:35:50.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:35:50.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:35:50.482 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:35:50.482 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:35:50.482 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:35:50.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:35:50.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:35:50.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:35:50.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:35:50.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:35:50.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:35:50.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:35:50.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:35:50.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:35:50.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:35:50.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-17T11:35:50.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:35:50.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:35:50.483 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:35:50.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T11:35:50.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:35:50.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:35:50.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:35:50.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:35:50.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:35:50.495 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:50.488+0000 7f8926ac38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:50.496 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:50.492+0000 7f8926ac38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:50.497 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:50.492+0000 7f8926ac38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:35:50.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:35:50.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:35:50.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:35:50.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:35:50.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:35:50.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:50.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:35:50.633 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:35:50.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:50.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:35:50.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:51.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:51.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:51.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:35:51.783 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:35:51.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:51.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:35:51.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:51.978 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:51.972+0000 7f8926ac38c0 -1 Falling back to public interface 2026-02-17T11:35:52.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:52.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:52.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:35:52.934 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:35:52.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:52.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:35:53.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:54.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:54.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:54.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:35:54.082 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:35:54.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:54.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:35:54.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:35:55.020 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:35:55.012+0000 7f8926ac38c0 -1 osd.0 19 log_to_monitors true 2026-02-17T11:35:55.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:35:55.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:35:55.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:35:55.233 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:35:55.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:35:55.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:35:55.383 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 23 up_thru 0 down_at 20 last_clean_interval [5,19) [v2:127.0.0.1:6802/3207071761,v1:127.0.0.1:6803/3207071761] [v2:127.0.0.1:6804/3207071761,v1:127.0.0.1:6805/3207071761] exists,up 0f717407-e69b-4661-baba-016b4f6edfe2 2026-02-17T11:35:55.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:35:55.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:35:55.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:35:55.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:35:55.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:35:55.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:35:55.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:35:55.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:35:55.384 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:35:55.384 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:35:55.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:35:55.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:35:55.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:35:55.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:35:55.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:35:55.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:35:55.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:35:55.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:35:55.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:35:55.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:35:55.641 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:35:55.642 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:35:55.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:35:55.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:35:55.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:35:55.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247810 2026-02-17T11:35:55.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247810 2026-02-17T11:35:55.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810' 2026-02-17T11:35:55.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:35:55.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:35:55.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705670 2026-02-17T11:35:55.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705670 2026-02-17T11:35:55.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810 1-38654705670' 2026-02-17T11:35:55.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:35:55.784 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:35:55.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574853 2026-02-17T11:35:55.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574853 2026-02-17T11:35:55.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810 1-38654705670 2-55834574853' 2026-02-17T11:35:55.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:35:55.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247810 2026-02-17T11:35:55.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:35:55.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:35:55.858 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247810 2026-02-17T11:35:55.858 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:35:55.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247810 2026-02-17T11:35:55.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247810' 2026-02-17T11:35:55.859 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 98784247810 2026-02-17T11:35:55.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:35:56.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247810 -lt 98784247810 2026-02-17T11:35:56.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:35:56.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705670 2026-02-17T11:35:56.027 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:35:56.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:35:56.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705670 2026-02-17T11:35:56.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:35:56.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705670 2026-02-17T11:35:56.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-17T11:35:56.031 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705670 2026-02-17T11:35:56.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:35:56.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705671 -lt 38654705670 2026-02-17T11:35:56.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:35:56.189 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574853 2026-02-17T11:35:56.189 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:35:56.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:35:56.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574853 2026-02-17T11:35:56.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:35:56.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574853 2026-02-17T11:35:56.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574853' 2026-02-17T11:35:56.192 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 55834574853 2026-02-17T11:35:56.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:35:56.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574853 -lt 55834574853 2026-02-17T11:35:56.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:35:56.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:35:56.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:35:56.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:35:56.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:35:56.538 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:35:56.538 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:35:56.538 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:35:56.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:35:56.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:35:56.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:35:56.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:35:56.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:35:56.693 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:35:56.693 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:35:56.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:35:56.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:35:56.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:35:56.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:413: TEST_auto_repair_bluestore_tag: get_pg testpool SOMETHING 2026-02-17T11:35:56.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=testpool 2026-02-17T11:35:56.875 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:35:56.875 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map testpool SOMETHING 2026-02-17T11:35:56.875 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:35:57.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:413: TEST_auto_repair_bluestore_tag: local pgid=1.0 2026-02-17T11:35:57.025 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:414: TEST_auto_repair_bluestore_tag: get_primary testpool SOMETHING 2026-02-17T11:35:57.025 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:35:57.025 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:35:57.025 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:35:57.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:35:57.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:414: TEST_auto_repair_bluestore_tag: local primary=1 2026-02-17T11:35:57.175 INFO:tasks.workunit.client.0.trial128.stdout:Affected PG 1.0 w/ primary 1 2026-02-17T11:35:57.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:415: TEST_auto_repair_bluestore_tag: echo 'Affected PG ' 1.0 ' w/ primary ' 1 2026-02-17T11:35:57.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:416: TEST_auto_repair_bluestore_tag: get_last_scrub_stamp 1.0 2026-02-17T11:35:57.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:35:57.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:35:57.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:35:57.177 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:35:57.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:416: TEST_auto_repair_bluestore_tag: local last_scrub_stamp=2026-02-17T11:35:43.773580+0000 2026-02-17T11:35:57.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:417: TEST_auto_repair_bluestore_tag: initiate_and_fetch_state 1 1.0 3.0 2026-02-17T11:35:57.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:322: initiate_and_fetch_state: local the_osd=osd.1 2026-02-17T11:35:57.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:323: initiate_and_fetch_state: local pgid=1.0 2026-02-17T11:35:57.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:324: initiate_and_fetch_state: get_last_scrub_stamp 1.0 2026-02-17T11:35:57.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T11:35:57.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:35:57.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:35:57.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T11:35:57.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:324: initiate_and_fetch_state: local last_scrub=2026-02-17T11:35:43.773580+0000 2026-02-17T11:35:57.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:326: initiate_and_fetch_state: set_config osd 1 osd_scrub_sleep 3.0 2026-02-17T11:35:57.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T11:35:57.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T11:35:57.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=osd_scrub_sleep 2026-02-17T11:35:57.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=3.0 2026-02-17T11:35:57.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T11:35:57.471 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T11:35:57.471 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T11:35:57.471 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T11:35:57.471 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:35:57.471 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:35:57.471 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:35:57.471 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.1.asok 2026-02-17T11:35:57.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40886/ceph-osd.1.asok config set osd_scrub_sleep 3.0 2026-02-17T11:35:57.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T11:35:57.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:327: initiate_and_fetch_state: set_config osd 1 osd_scrub_auto_repair true 2026-02-17T11:35:57.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T11:35:57.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T11:35:57.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=osd_scrub_auto_repair 2026-02-17T11:35:57.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T11:35:57.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T11:35:57.531 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T11:35:57.531 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T11:35:57.531 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T11:35:57.531 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:35:57.531 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:35:57.531 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:35:57.531 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.1.asok 2026-02-17T11:35:57.531 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40886/ceph-osd.1.asok config set osd_scrub_auto_repair true 2026-02-17T11:35:57.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T11:35:57.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:329: initiate_and_fetch_state: flush_pg_stats 2026-02-17T11:35:57.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:35:57.588 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:35:57.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:35:57.742 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:35:57.742 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:35:57.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:35:57.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:35:57.742 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:35:57.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247811 2026-02-17T11:35:57.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247811 2026-02-17T11:35:57.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247811' 2026-02-17T11:35:57.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:35:57.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:35:57.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705672 2026-02-17T11:35:57.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705672 2026-02-17T11:35:57.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247811 1-38654705672' 2026-02-17T11:35:57.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:35:57.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:35:57.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574854 2026-02-17T11:35:57.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574854 2026-02-17T11:35:57.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247811 1-38654705672 2-55834574854' 2026-02-17T11:35:57.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:35:57.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247811 2026-02-17T11:35:57.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:35:57.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:35:57.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247811 2026-02-17T11:35:57.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:35:57.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247811 2026-02-17T11:35:57.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247811' 2026-02-17T11:35:57.962 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 98784247811 2026-02-17T11:35:57.962 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:35:58.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247811 -lt 98784247811 2026-02-17T11:35:58.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:35:58.120 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705672 2026-02-17T11:35:58.120 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:35:58.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:35:58.121 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705672 2026-02-17T11:35:58.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:35:58.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705672 2026-02-17T11:35:58.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705672' 2026-02-17T11:35:58.124 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705672 2026-02-17T11:35:58.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:35:58.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705672 -lt 38654705672 2026-02-17T11:35:58.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:35:58.284 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574854 2026-02-17T11:35:58.284 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:35:58.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:35:58.286 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574854 2026-02-17T11:35:58.286 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:35:58.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574854 2026-02-17T11:35:58.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574854' 2026-02-17T11:35:58.288 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574854 2026-02-17T11:35:58.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:35:58.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574853 -lt 55834574854 2026-02-17T11:35:58.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:35:59.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:35:59.454 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:35:59.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574853 -lt 55834574854 2026-02-17T11:35:59.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:36:00.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:36:00.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:36:00.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574855 -lt 55834574854 2026-02-17T11:36:00.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:330: initiate_and_fetch_state: date --rfc-3339=ns 2026-02-17T11:36:00.764 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17 11:36:00.761184820+00:00 2026-02-17T11:36:00.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:333: initiate_and_fetch_state: get_asok_path osd.1 2026-02-17T11:36:00.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T11:36:00.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T11:36:00.765 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:36:00.765 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:36:00.766 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:36:00.766 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.1.asok 2026-02-17T11:36:00.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:333: initiate_and_fetch_state: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40886/ceph-osd.1.asok schedule-deep-scrub 1.0 2026-02-17T11:36:00.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i=0 )) 2026-02-17T11:36:00.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:00.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:00.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:00.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:00.904 INFO:tasks.workunit.client.0.trial128.stdout:{"deep":true,"must":false,"stamp":"2026-02-03T11:34:20.820489+0000"}0 ) state now: "active+clean" 2026-02-17T11:36:00.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 0 ') state now: ' '"active+clean"' 2026-02-17T11:36:00.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:00.905 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:00.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:00.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 0 == 4 ']' 2026-02-17T11:36:00.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:01.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:01.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:01.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:01.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:01.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:01.280 INFO:tasks.workunit.client.0.trial128.stdout:1 ) state now: "active+clean" 2026-02-17T11:36:01.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 1 ') state now: ' '"active+clean"' 2026-02-17T11:36:01.281 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:01.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:01.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:01.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 1 == 4 ']' 2026-02-17T11:36:01.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:01.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:01.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:01.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:01.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:01.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:01.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 2 ') state now: ' '"active+clean"' 2026-02-17T11:36:01.655 INFO:tasks.workunit.client.0.trial128.stdout:2 ) state now: "active+clean" 2026-02-17T11:36:01.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:01.656 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:01.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:01.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 2 == 4 ']' 2026-02-17T11:36:01.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:01.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:01.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:01.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:01.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:02.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:02.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 3 ') state now: ' '"active+clean"' 2026-02-17T11:36:02.030 INFO:tasks.workunit.client.0.trial128.stdout:3 ) state now: "active+clean" 2026-02-17T11:36:02.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:02.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:02.031 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:02.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 3 == 4 ']' 2026-02-17T11:36:02.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:02.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:02.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:02.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:02.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:02.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:02.405 INFO:tasks.workunit.client.0.trial128.stdout:4 ) state now: "active+clean" 2026-02-17T11:36:02.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 4 ') state now: ' '"active+clean"' 2026-02-17T11:36:02.405 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:02.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:02.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:02.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 4 == 4 ']' 2026-02-17T11:36:02.406 INFO:tasks.workunit.client.0.trial128.stdout:loop --------> 4 2026-02-17T11:36:02.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:350: initiate_and_fetch_state: echo 'loop --------> ' 4 2026-02-17T11:36:02.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:02.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:02.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:02.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:02.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:02.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:02.783 INFO:tasks.workunit.client.0.trial128.stdout:5 ) state now: "active+clean" 2026-02-17T11:36:02.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 5 ') state now: ' '"active+clean"' 2026-02-17T11:36:02.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:02.784 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:02.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:02.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 5 == 4 ']' 2026-02-17T11:36:02.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:03.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:03.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:03.085 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:03.085 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:03.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:03.157 INFO:tasks.workunit.client.0.trial128.stdout:6 ) state now: "active+clean" 2026-02-17T11:36:03.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 6 ') state now: ' '"active+clean"' 2026-02-17T11:36:03.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:03.157 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:03.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:03.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 6 == 4 ']' 2026-02-17T11:36:03.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:03.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:03.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:03.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:03.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:03.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:03.531 INFO:tasks.workunit.client.0.trial128.stdout:7 ) state now: "active+clean" 2026-02-17T11:36:03.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 7 ') state now: ' '"active+clean"' 2026-02-17T11:36:03.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:03.532 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:03.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:03.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 7 == 4 ']' 2026-02-17T11:36:03.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:03.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:03.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:03.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:03.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:03.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:03.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 8 ') state now: ' '"active+clean"' 2026-02-17T11:36:03.904 INFO:tasks.workunit.client.0.trial128.stdout:8 ) state now: "active+clean" 2026-02-17T11:36:03.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:03.904 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:03.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:03.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 8 == 4 ']' 2026-02-17T11:36:03.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:04.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:04.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:04.205 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:04.205 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:04.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:04.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 9 ') state now: ' '"active+clean"' 2026-02-17T11:36:04.277 INFO:tasks.workunit.client.0.trial128.stdout:9 ) state now: "active+clean" 2026-02-17T11:36:04.277 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:04.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:04.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:04.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 9 == 4 ']' 2026-02-17T11:36:04.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:04.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:04.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:04.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:04.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:04.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:04.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 10 ') state now: ' '"active+clean"' 2026-02-17T11:36:04.651 INFO:tasks.workunit.client.0.trial128.stdout:10 ) state now: "active+clean" 2026-02-17T11:36:04.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:04.652 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:04.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:04.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 0 == 4 ']' 2026-02-17T11:36:04.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:04.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:04.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:04.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:04.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:05.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:05.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 11 ') state now: ' '"active+clean"' 2026-02-17T11:36:05.024 INFO:tasks.workunit.client.0.trial128.stdout:11 ) state now: "active+clean" 2026-02-17T11:36:05.024 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:05.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:05.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:05.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 1 == 4 ']' 2026-02-17T11:36:05.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:05.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:05.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:05.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:05.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:05.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:05.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 12 ') state now: ' '"active+clean"' 2026-02-17T11:36:05.400 INFO:tasks.workunit.client.0.trial128.stdout:12 ) state now: "active+clean" 2026-02-17T11:36:05.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:05.400 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:05.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:05.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 2 == 4 ']' 2026-02-17T11:36:05.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:05.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:05.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:05.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:05.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:05.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:05.771 INFO:tasks.workunit.client.0.trial128.stdout:13 ) state now: "active+clean" 2026-02-17T11:36:05.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 13 ') state now: ' '"active+clean"' 2026-02-17T11:36:05.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:05.772 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:05.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:05.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 3 == 4 ']' 2026-02-17T11:36:05.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:06.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:06.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:06.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:06.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:06.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:06.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 14 ') state now: ' '"active+clean"' 2026-02-17T11:36:06.144 INFO:tasks.workunit.client.0.trial128.stdout:14 ) state now: "active+clean" 2026-02-17T11:36:06.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:06.144 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:06.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:06.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 4 == 4 ']' 2026-02-17T11:36:06.145 INFO:tasks.workunit.client.0.trial128.stdout:loop --------> 14 2026-02-17T11:36:06.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:350: initiate_and_fetch_state: echo 'loop --------> ' 14 2026-02-17T11:36:06.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:06.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:06.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:06.445 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:06.445 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:06.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:06.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 15 ') state now: ' '"active+clean"' 2026-02-17T11:36:06.518 INFO:tasks.workunit.client.0.trial128.stdout:15 ) state now: "active+clean" 2026-02-17T11:36:06.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:06.518 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:06.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:06.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 5 == 4 ']' 2026-02-17T11:36:06.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:06.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:06.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:06.819 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:06.819 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:06.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:06.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 16 ') state now: ' '"active+clean"' 2026-02-17T11:36:06.891 INFO:tasks.workunit.client.0.trial128.stdout:16 ) state now: "active+clean" 2026-02-17T11:36:06.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:06.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:06.892 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:06.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 6 == 4 ']' 2026-02-17T11:36:06.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:07.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:07.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:07.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:07.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:07.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:07.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 17 ') state now: ' '"active+clean"' 2026-02-17T11:36:07.263 INFO:tasks.workunit.client.0.trial128.stdout:17 ) state now: "active+clean" 2026-02-17T11:36:07.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:07.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:07.264 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:07.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 7 == 4 ']' 2026-02-17T11:36:07.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:07.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:07.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:07.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:07.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:07.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:07.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 18 ') state now: ' '"active+clean"' 2026-02-17T11:36:07.636 INFO:tasks.workunit.client.0.trial128.stdout:18 ) state now: "active+clean" 2026-02-17T11:36:07.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:07.637 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:07.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:07.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 8 == 4 ']' 2026-02-17T11:36:07.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:07.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:07.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:07.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:07.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:08.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:08.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 19 ') state now: ' '"active+clean"' 2026-02-17T11:36:08.010 INFO:tasks.workunit.client.0.trial128.stdout:19 ) state now: "active+clean" 2026-02-17T11:36:08.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:08.011 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:08.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:08.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 9 == 4 ']' 2026-02-17T11:36:08.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:08.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:08.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:08.312 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:08.312 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:08.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:08.383 INFO:tasks.workunit.client.0.trial128.stdout:20 ) state now: "active+clean" 2026-02-17T11:36:08.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 20 ') state now: ' '"active+clean"' 2026-02-17T11:36:08.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:08.384 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:08.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:08.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 0 == 4 ']' 2026-02-17T11:36:08.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:08.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:08.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:08.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:08.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:08.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:08.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 21 ') state now: ' '"active+clean"' 2026-02-17T11:36:08.759 INFO:tasks.workunit.client.0.trial128.stdout:21 ) state now: "active+clean" 2026-02-17T11:36:08.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:08.760 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:08.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:08.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 1 == 4 ']' 2026-02-17T11:36:08.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:09.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:09.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:09.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:09.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:09.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:09.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 22 ') state now: ' '"active+clean"' 2026-02-17T11:36:09.137 INFO:tasks.workunit.client.0.trial128.stdout:22 ) state now: "active+clean" 2026-02-17T11:36:09.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:09.138 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:09.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:09.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 2 == 4 ']' 2026-02-17T11:36:09.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:09.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:09.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:09.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:09.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:09.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:09.512 INFO:tasks.workunit.client.0.trial128.stdout:23 ) state now: "active+clean" 2026-02-17T11:36:09.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 23 ') state now: ' '"active+clean"' 2026-02-17T11:36:09.512 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:09.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:09.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:09.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 3 == 4 ']' 2026-02-17T11:36:09.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:09.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:09.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:09.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:09.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:09.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:09.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 24 ') state now: ' '"active+clean"' 2026-02-17T11:36:09.888 INFO:tasks.workunit.client.0.trial128.stdout:24 ) state now: "active+clean" 2026-02-17T11:36:09.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:09.889 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:09.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:09.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 4 == 4 ']' 2026-02-17T11:36:09.890 INFO:tasks.workunit.client.0.trial128.stdout:loop --------> 24 2026-02-17T11:36:09.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:350: initiate_and_fetch_state: echo 'loop --------> ' 24 2026-02-17T11:36:09.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:10.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:10.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:10.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:10.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:10.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:10.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 25 ') state now: ' '"active+clean"' 2026-02-17T11:36:10.265 INFO:tasks.workunit.client.0.trial128.stdout:25 ) state now: "active+clean" 2026-02-17T11:36:10.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:10.265 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:10.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:10.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 5 == 4 ']' 2026-02-17T11:36:10.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:10.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:10.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:10.566 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:10.566 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:10.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:10.643 INFO:tasks.workunit.client.0.trial128.stdout:26 ) state now: "active+clean" 2026-02-17T11:36:10.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 26 ') state now: ' '"active+clean"' 2026-02-17T11:36:10.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:10.643 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:10.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:10.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 6 == 4 ']' 2026-02-17T11:36:10.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:10.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:10.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:10.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:10.946 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:11.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:11.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 27 ') state now: ' '"active+clean"' 2026-02-17T11:36:11.018 INFO:tasks.workunit.client.0.trial128.stdout:27 ) state now: "active+clean" 2026-02-17T11:36:11.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:11.019 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:11.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:11.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 7 == 4 ']' 2026-02-17T11:36:11.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:11.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:11.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:11.319 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:11.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:11.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean"' 2026-02-17T11:36:11.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 28 ') state now: ' '"active+clean"' 2026-02-17T11:36:11.392 INFO:tasks.workunit.client.0.trial128.stdout:28 ) state now: "active+clean" 2026-02-17T11:36:11.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:11.393 INFO:tasks.workunit.client.0.trial128.stdout:"active+clean" 2026-02-17T11:36:11.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:346: initiate_and_fetch_state: echo '"active+clean"' 2026-02-17T11:36:11.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:349: initiate_and_fetch_state: '[' 8 == 4 ']' 2026-02-17T11:36:11.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:352: initiate_and_fetch_state: sleep 0.3 2026-02-17T11:36:11.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i++ )) 2026-02-17T11:36:11.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:336: initiate_and_fetch_state: (( i < 80 )) 2026-02-17T11:36:11.693 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: ceph pg 1.0 query --format json 2026-02-17T11:36:11.693 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: jq .state 2026-02-17T11:36:11.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:338: initiate_and_fetch_state: st='"active+clean+scrubbing+deep"' 2026-02-17T11:36:11.765 INFO:tasks.workunit.client.0.trial128.stdout:29 ) state now: "active+clean+scrubbing+deep" 2026-02-17T11:36:11.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:339: initiate_and_fetch_state: echo 29 ') state now: ' '"active+clean+scrubbing+deep"' 2026-02-17T11:36:11.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:341: initiate_and_fetch_state: case "$st" in 2026-02-17T11:36:11.766 INFO:tasks.workunit.client.0.trial128.stdout:found scrub 2026-02-17T11:36:11.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:343: initiate_and_fetch_state: echo 'found scrub' 2026-02-17T11:36:11.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:343: initiate_and_fetch_state: return 0 2026-02-17T11:36:11.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:418: TEST_auto_repair_bluestore_tag: r=0 2026-02-17T11:36:11.767 INFO:tasks.workunit.client.0.trial128.stdout:initiate_and_fetch_state ret: 0 2026-02-17T11:36:11.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:419: TEST_auto_repair_bluestore_tag: echo 'initiate_and_fetch_state ret: ' 0 2026-02-17T11:36:11.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:420: TEST_auto_repair_bluestore_tag: set_config osd td/osd-scrub-repair osd_scrub_sleep 0 2026-02-17T11:36:11.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T11:36:11.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=td/osd-scrub-repair 2026-02-17T11:36:11.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=osd_scrub_sleep 2026-02-17T11:36:11.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=0 2026-02-17T11:36:11.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T11:36:11.769 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.td/osd-scrub-repair 2026-02-17T11:36:11.769 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.td/osd-scrub-repair 2026-02-17T11:36:11.769 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.td/osd-scrub-repair ']' 2026-02-17T11:36:11.769 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:36:11.770 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:36:11.770 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:36:11.770 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.td/osd-scrub-repair.asok 2026-02-17T11:36:11.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40886/ceph-osd.td/osd-scrub-repair.asok config set osd_scrub_sleep 0 2026-02-17T11:36:11.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test == true 2026-02-17T11:36:11.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh: line 1168: test: ==: unary operator expected 2026-02-17T11:36:11.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:421: TEST_auto_repair_bluestore_tag: '[' 0 -ne 0 ']' 2026-02-17T11:36:11.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:425: TEST_auto_repair_bluestore_tag: wait_end_of_scrub 1 1.0 2026-02-17T11:36:11.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:360: wait_end_of_scrub: local the_osd=osd.1 2026-02-17T11:36:11.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:361: wait_end_of_scrub: local pgid=1.0 2026-02-17T11:36:11.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i=0 )) 2026-02-17T11:36:11.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i < 40 )) 2026-02-17T11:36:11.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-17T11:36:11.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: jq .state 2026-02-17T11:36:11.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: st='"active+clean+scrubbing+deep"' 2026-02-17T11:36:11.884 INFO:tasks.workunit.client.0.trial128.stdout:wait-scrub-end state now: "active+clean+scrubbing+deep" 2026-02-17T11:36:11.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:365: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean+scrubbing+deep"' 2026-02-17T11:36:11.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:366: wait_end_of_scrub: [[ "active+clean+scrubbing+deep" =~ (.*scrubbing.*) ]] 2026-02-17T11:36:11.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:367: wait_end_of_scrub: '[' 0 == 4 ']' 2026-02-17T11:36:11.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:370: wait_end_of_scrub: sleep 0.3 2026-02-17T11:36:12.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i++ )) 2026-02-17T11:36:12.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i < 40 )) 2026-02-17T11:36:12.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-17T11:36:12.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: jq .state 2026-02-17T11:36:12.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: st='"active+clean+scrubbing+deep"' 2026-02-17T11:36:12.256 INFO:tasks.workunit.client.0.trial128.stdout:wait-scrub-end state now: "active+clean+scrubbing+deep" 2026-02-17T11:36:12.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:365: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean+scrubbing+deep"' 2026-02-17T11:36:12.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:366: wait_end_of_scrub: [[ "active+clean+scrubbing+deep" =~ (.*scrubbing.*) ]] 2026-02-17T11:36:12.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:367: wait_end_of_scrub: '[' 1 == 4 ']' 2026-02-17T11:36:12.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:370: wait_end_of_scrub: sleep 0.3 2026-02-17T11:36:12.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i++ )) 2026-02-17T11:36:12.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i < 40 )) 2026-02-17T11:36:12.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-17T11:36:12.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: jq .state 2026-02-17T11:36:12.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: st='"active+clean+scrubbing+deep"' 2026-02-17T11:36:12.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:365: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean+scrubbing+deep"' 2026-02-17T11:36:12.629 INFO:tasks.workunit.client.0.trial128.stdout:wait-scrub-end state now: "active+clean+scrubbing+deep" 2026-02-17T11:36:12.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:366: wait_end_of_scrub: [[ "active+clean+scrubbing+deep" =~ (.*scrubbing.*) ]] 2026-02-17T11:36:12.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:367: wait_end_of_scrub: '[' 2 == 4 ']' 2026-02-17T11:36:12.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:370: wait_end_of_scrub: sleep 0.3 2026-02-17T11:36:12.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i++ )) 2026-02-17T11:36:12.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i < 40 )) 2026-02-17T11:36:12.931 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-17T11:36:12.931 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: jq .state 2026-02-17T11:36:13.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: st='"active+clean+scrubbing+deep"' 2026-02-17T11:36:13.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:365: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean+scrubbing+deep"' 2026-02-17T11:36:13.006 INFO:tasks.workunit.client.0.trial128.stdout:wait-scrub-end state now: "active+clean+scrubbing+deep" 2026-02-17T11:36:13.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:366: wait_end_of_scrub: [[ "active+clean+scrubbing+deep" =~ (.*scrubbing.*) ]] 2026-02-17T11:36:13.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:367: wait_end_of_scrub: '[' 3 == 4 ']' 2026-02-17T11:36:13.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:370: wait_end_of_scrub: sleep 0.3 2026-02-17T11:36:13.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i++ )) 2026-02-17T11:36:13.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i < 40 )) 2026-02-17T11:36:13.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-17T11:36:13.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: jq .state 2026-02-17T11:36:13.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: st='"active+clean+scrubbing+deep"' 2026-02-17T11:36:13.381 INFO:tasks.workunit.client.0.trial128.stdout:wait-scrub-end state now: "active+clean+scrubbing+deep" 2026-02-17T11:36:13.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:365: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean+scrubbing+deep"' 2026-02-17T11:36:13.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:366: wait_end_of_scrub: [[ "active+clean+scrubbing+deep" =~ (.*scrubbing.*) ]] 2026-02-17T11:36:13.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:367: wait_end_of_scrub: '[' 4 == 4 ']' 2026-02-17T11:36:13.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:368: wait_end_of_scrub: flush_pg_stats 2026-02-17T11:36:13.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:36:13.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:36:13.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:36:13.541 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:36:13.541 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:36:13.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:36:13.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:36:13.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:36:13.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247815 2026-02-17T11:36:13.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247815 2026-02-17T11:36:13.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247815' 2026-02-17T11:36:13.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:36:13.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:36:13.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705676 2026-02-17T11:36:13.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705676 2026-02-17T11:36:13.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247815 1-38654705676' 2026-02-17T11:36:13.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:36:13.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:36:13.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574858 2026-02-17T11:36:13.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574858 2026-02-17T11:36:13.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247815 1-38654705676 2-55834574858' 2026-02-17T11:36:13.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:36:13.754 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247815 2026-02-17T11:36:13.755 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:36:13.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:36:13.756 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247815 2026-02-17T11:36:13.757 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:36:13.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247815 2026-02-17T11:36:13.758 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 98784247815 2026-02-17T11:36:13.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247815' 2026-02-17T11:36:13.759 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:36:13.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247814 -lt 98784247815 2026-02-17T11:36:13.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:36:14.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:36:14.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:36:15.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247815 -lt 98784247815 2026-02-17T11:36:15.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:36:15.078 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705676 2026-02-17T11:36:15.078 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:36:15.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:36:15.079 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705676 2026-02-17T11:36:15.079 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:36:15.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705676 2026-02-17T11:36:15.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705676' 2026-02-17T11:36:15.081 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705676 2026-02-17T11:36:15.082 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:36:15.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705676 -lt 38654705676 2026-02-17T11:36:15.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:36:15.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574858 2026-02-17T11:36:15.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:36:15.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:36:15.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574858 2026-02-17T11:36:15.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:36:15.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574858 2026-02-17T11:36:15.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574858' 2026-02-17T11:36:15.235 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574858 2026-02-17T11:36:15.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:36:15.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574858 -lt 55834574858 2026-02-17T11:36:15.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:370: wait_end_of_scrub: sleep 0.3 2026-02-17T11:36:15.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i++ )) 2026-02-17T11:36:15.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:363: wait_end_of_scrub: (( i < 40 )) 2026-02-17T11:36:15.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: ceph pg 1.0 query --format json 2026-02-17T11:36:15.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: jq .state 2026-02-17T11:36:15.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:364: wait_end_of_scrub: st='"active+clean"' 2026-02-17T11:36:15.776 INFO:tasks.workunit.client.0.trial128.stdout:wait-scrub-end state now: "active+clean" 2026-02-17T11:36:15.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:365: wait_end_of_scrub: echo 'wait-scrub-end state now: ' '"active+clean"' 2026-02-17T11:36:15.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:366: wait_end_of_scrub: [[ "active+clean" =~ (.*scrubbing.*) ]] 2026-02-17T11:36:15.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:366: wait_end_of_scrub: break 2026-02-17T11:36:15.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:373: wait_end_of_scrub: [[ "active+clean" =~ (.*scrubbing.*) ]] 2026-02-17T11:36:15.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:378: wait_end_of_scrub: return 0 2026-02-17T11:36:15.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:426: TEST_auto_repair_bluestore_tag: ceph pg dump pgs 2026-02-17T11:36:15.917 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T11:36:15.917 INFO:tasks.workunit.client.0.trial128.stdout:1.0 1 0 0 0 0 7 0 0 1 0 1 active+clean+scrubbing+deep 2026-02-17T11:36:11.468679+0000 19'1 25:49 [1,0] 1 [1,0] 1 0'0 2026-02-03T11:34:20.820489+0000 0'0 2026-02-03T11:34:20.820489+0000 0 0 deep scrubbing for 3s 0 0 2026-02-17T11:36:15.917 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T11:36:15.917 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T11:36:15.917 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T11:36:15.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:430: TEST_auto_repair_bluestore_tag: get_not_primary testpool SOMETHING 2026-02-17T11:36:15.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T11:36:15.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:36:15.926 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T11:36:15.926 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:36:15.926 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:36:15.927 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:36:15.927 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:36:16.074 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T11:36:16.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:36:16.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T11:36:16.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:430: TEST_auto_repair_bluestore_tag: objectstore_tool td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T11:36:16.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:36:16.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:36:16.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:36:16.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:36:16.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T11:36:16.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:36:16.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:36:16.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:36:16.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:36:16.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:36:16.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:36:16.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:36:16.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:36:16.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:36:16.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:36:17.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:36:17.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T11:36:17.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:36:17.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:36:17.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:36:17.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:36:17.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:36:17.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING list-attrs 2026-02-17T11:36:17.852 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:36:17.852 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:36:18.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T11:36:18.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:36:18.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:36:18.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:36:18.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:36:18.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:36:18.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:36:18.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:36:18.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:36:18.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:36:18.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:36:18.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:36:18.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:36:18.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:36:18.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:36:18.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:36:18.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:36:18.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:36:18.389 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:36:18.389 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:36:18.389 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:36:18.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:36:18.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:36:18.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:36:18.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:36:18.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:36:18.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:36:18.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:36:18.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:36:18.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:36:18.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:36:18.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-17T11:36:18.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:36:18.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:36:18.393 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:36:18.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T11:36:18.393 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:36:18.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:36:18.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:36:18.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:36:18.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:36:18.409 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:18.403+0000 7fa1401848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:18.410 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:18.403+0000 7fa1401848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:18.411 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:18.403+0000 7fa1401848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:18.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:36:18.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:36:18.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:36:18.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:36:18.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:36:18.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:18.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:36:18.545 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:36:18.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:18.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:36:18.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:19.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:19.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:19.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:36:19.701 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:36:19.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:19.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:36:19.710 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:19.703+0000 7fa1401848c0 -1 Falling back to public interface 2026-02-17T11:36:19.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:20.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:20.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:20.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:36:20.856 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:36:20.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:20.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:36:21.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:22.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:22.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:22.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:36:22.011 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:36:22.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:22.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:36:22.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:22.674 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:22.667+0000 7fa1401848c0 -1 osd.0 25 log_to_monitors true 2026-02-17T11:36:23.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:23.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:23.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:36:23.166 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:36:23.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:23.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:36:23.321 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 29 up_thru 0 down_at 26 last_clean_interval [23,25) [v2:127.0.0.1:6802/979929362,v1:127.0.0.1:6803/979929362] [v2:127.0.0.1:6804/979929362,v1:127.0.0.1:6805/979929362] exists,up 0f717407-e69b-4661-baba-016b4f6edfe2 2026-02-17T11:36:23.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:36:23.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:36:23.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:36:23.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:36:23.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:36:23.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:36:23.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:36:23.322 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:36:23.323 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:36:23.323 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:36:23.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:36:23.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:36:23.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:36:23.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:36:23.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:36:23.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:36:23.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:36:23.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:36:23.424 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:36:23.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:36:23.579 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:36:23.579 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:36:23.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:36:23.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:36:23.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:36:23.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051586 2026-02-17T11:36:23.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051586 2026-02-17T11:36:23.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586' 2026-02-17T11:36:23.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:36:23.653 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:36:23.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705679 2026-02-17T11:36:23.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705679 2026-02-17T11:36:23.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586 1-38654705679' 2026-02-17T11:36:23.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:36:23.723 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:36:23.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574861 2026-02-17T11:36:23.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574861 2026-02-17T11:36:23.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-124554051586 1-38654705679 2-55834574861' 2026-02-17T11:36:23.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:36:23.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-124554051586 2026-02-17T11:36:23.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:36:23.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:36:23.798 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-124554051586 2026-02-17T11:36:23.798 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:36:23.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051586 2026-02-17T11:36:23.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 124554051586' 2026-02-17T11:36:23.799 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 124554051586 2026-02-17T11:36:23.800 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:36:23.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247816 -lt 124554051586 2026-02-17T11:36:23.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:36:24.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:36:24.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:36:25.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051586 -lt 124554051586 2026-02-17T11:36:25.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:36:25.120 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705679 2026-02-17T11:36:25.120 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:36:25.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:36:25.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705679 2026-02-17T11:36:25.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:36:25.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705679 2026-02-17T11:36:25.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705679' 2026-02-17T11:36:25.124 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705679 2026-02-17T11:36:25.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:36:25.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705679 -lt 38654705679 2026-02-17T11:36:25.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:36:25.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574861 2026-02-17T11:36:25.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:36:25.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:36:25.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574861 2026-02-17T11:36:25.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:36:25.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574861 2026-02-17T11:36:25.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574861' 2026-02-17T11:36:25.292 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 55834574861 2026-02-17T11:36:25.292 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:36:25.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574861 -lt 55834574861 2026-02-17T11:36:25.456 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:36:25.456 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:36:25.456 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:36:25.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:36:25.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:36:25.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:36:25.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:36:25.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:36:25.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:36:25.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:36:25.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:36:25.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:36:25.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:36:25.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:36:25.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:36:25.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:36:25.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:36:25.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:36:25.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:431: TEST_auto_repair_bluestore_tag: get_not_primary testpool SOMETHING 2026-02-17T11:36:25.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T11:36:25.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:36:25.982 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool SOMETHING 2026-02-17T11:36:25.982 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T11:36:25.982 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:36:25.983 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:36:25.983 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:36:26.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T11:36:26.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool SOMETHING 2026-02-17T11:36:26.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T11:36:26.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:431: TEST_auto_repair_bluestore_tag: objectstore_tool td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-17T11:36:26.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:36:26.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:36:26.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:36:26.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:36:26.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-17T11:36:26.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:36:26.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:36:26.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:36:26.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:36:26.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:36:26.292 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:36:26.292 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:36:26.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:36:26.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:36:26.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:36:27.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:36:27.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-17T11:36:27.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:36:27.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:36:27.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:36:27.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:36:27.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:36:27.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING get-bytes td/osd-scrub-repair/COPY 2026-02-17T11:36:28.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T11:36:28.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:36:28.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:36:28.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:36:28.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:36:28.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:36:28.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:36:28.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:36:28.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:36:28.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:36:28.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:36:28.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:36:28.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:36:28.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:36:28.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:36:28.457 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:36:28.457 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:36:28.457 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:36:28.457 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:36:28.457 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:36:28.457 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:36:28.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:36:28.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:36:28.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:36:28.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:36:28.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:36:28.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:36:28.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:36:28.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:36:28.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:36:28.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:36:28.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq' 2026-02-17T11:36:28.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:36:28.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:36:28.460 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:36:28.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --osd-op-queue=wpq 2026-02-17T11:36:28.460 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:36:28.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:36:28.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:36:28.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:36:28.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:36:28.481 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:28.475+0000 7fce3cd4b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:28.483 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:28.475+0000 7fce3cd4b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:28.483 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:28.475+0000 7fce3cd4b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:28.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:36:28.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:36:28.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:36:28.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:36:28.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:36:28.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:28.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:36:28.612 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:36:28.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:28.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:36:28.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:29.731 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:29.723+0000 7fce3cd4b8c0 -1 Falling back to public interface 2026-02-17T11:36:29.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:29.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:29.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:36:29.769 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:36:29.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:29.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:36:29.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:30.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:30.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:30.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:36:30.923 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:36:30.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:30.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:36:31.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:32.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:32.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:32.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:36:32.079 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:36:32.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:32.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:36:32.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:32.714 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:32.707+0000 7fce3cd4b8c0 -1 osd.0 30 log_to_monitors true 2026-02-17T11:36:33.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:33.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:33.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:36:33.230 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:36:33.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:33.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:36:33.385 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 34 up_thru 0 down_at 31 last_clean_interval [29,30) [v2:127.0.0.1:6802/2383085476,v1:127.0.0.1:6803/2383085476] [v2:127.0.0.1:6804/2383085476,v1:127.0.0.1:6805/2383085476] exists,up 0f717407-e69b-4661-baba-016b4f6edfe2 2026-02-17T11:36:33.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:36:33.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:36:33.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:36:33.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:36:33.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:36:33.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:36:33.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:36:33.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:36:33.387 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:36:33.387 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:36:33.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:36:33.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:36:33.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:36:33.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:36:33.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:36:33.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:36:33.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:36:33.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:36:33.489 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:36:33.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:36:33.649 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:36:33.649 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:36:33.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:36:33.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:36:33.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:36:33.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888066 2026-02-17T11:36:33.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888066 2026-02-17T11:36:33.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888066' 2026-02-17T11:36:33.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:36:33.723 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:36:33.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705682 2026-02-17T11:36:33.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705682 2026-02-17T11:36:33.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888066 1-38654705682' 2026-02-17T11:36:33.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:36:33.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:36:33.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574864 2026-02-17T11:36:33.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574864 2026-02-17T11:36:33.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888066 1-38654705682 2-55834574864' 2026-02-17T11:36:33.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:36:33.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-146028888066 2026-02-17T11:36:33.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:36:33.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:36:33.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-146028888066 2026-02-17T11:36:33.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:36:33.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888066 2026-02-17T11:36:33.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 146028888066' 2026-02-17T11:36:33.872 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 146028888066 2026-02-17T11:36:33.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:36:34.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888066 -lt 146028888066 2026-02-17T11:36:34.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:36:34.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705682 2026-02-17T11:36:34.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:36:34.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:36:34.039 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705682 2026-02-17T11:36:34.039 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:36:34.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705682 2026-02-17T11:36:34.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705682' 2026-02-17T11:36:34.041 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705682 2026-02-17T11:36:34.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:36:34.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705682 -lt 38654705682 2026-02-17T11:36:34.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:36:34.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574864 2026-02-17T11:36:34.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:36:34.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:36:34.204 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574864 2026-02-17T11:36:34.204 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:36:34.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574864 2026-02-17T11:36:34.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574864' 2026-02-17T11:36:34.206 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 55834574864 2026-02-17T11:36:34.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:36:34.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574864 -lt 55834574864 2026-02-17T11:36:34.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:36:34.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:36:34.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:36:34.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T11:36:34.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:36:34.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:36:34.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:36:34.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:36:34.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:36:34.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:36:34.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:36:34.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:36:34.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:36:34.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:36:34.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:36:34.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T11:36:34.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:36:34.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:36:34.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:432: TEST_auto_repair_bluestore_tag: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:36:34.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:433: TEST_auto_repair_bluestore_tag: grep scrub_finish td/osd-scrub-repair/osd.1.log 2026-02-17T11:36:34.900 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:36:14.467+0000 7f9a52192640 10 osd.1 pg_epoch: 25 pg[1.0( v 19'1 (0'0,19'1] local-lis/les=23/24 n=1 ec=14/14 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=19'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] planned AUTO_REPAIR TIME_FOR_DEEP] scrubber: scrub_finish before flags: AUTO_REPAIR. repair state: no-repair. deep_scrub_on_error: 0 2026-02-17T11:36:34.900 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:36:14.467+0000 7f9a52192640 10 osd.1 pg_epoch: 25 pg[1.0( v 19'1 (0'0,19'1] local-lis/les=23/24 n=1 ec=14/14 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=19'1 lcod 0'0 mlcod 0'0 active+clean+scrubbing+deep [ 1.0: AUTO_REPAIR ] ] scrubber: _scrub_finish info stats: valid m_is_repair: 1 2026-02-17T11:36:34.900 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:36:14.467+0000 7f9a52192640 15 osd.1 pg_epoch: 25 pg[1.0( v 19'1 (0'0,19'1] local-lis/les=23/24 n=1 ec=14/14 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=19'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish: 1 errors. 1 errors fixed 2026-02-17T11:36:34.900 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:36:14.467+0000 7f9a52192640 20 osd.1 pg_epoch: 25 pg[1.0( v 19'1 (0'0,19'1] local-lis/les=23/24 n=1 ec=14/14 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=19'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish All may be fixed 2026-02-17T11:36:34.900 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T11:36:14.467+0000 7f9a52192640 19 osd.1 pg_epoch: 25 pg[1.0( v 19'1 (0'0,19'1] local-lis/les=23/24 n=1 ec=14/14 lis/c=23/23 les/c/f=24/24/0 sis=23) [1,0] r=0 lpr=23 crt=19'1 lcod 0'0 mlcod 0'0 active+scrubbing+deep+repair [ 1.0: AUTO_REPAIR ] mbc={255={(1+0)=1}}] scrubber: scrub_finish shard 1 num_omap_bytes = 0 num_omap_keys = 0 2026-02-17T11:36:34.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T11:36:34.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:36:34.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:36:34.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:36:34.902 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:36:34.902 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:36:34.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:36:34.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:36:34.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:36:35.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:36:35.012 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:36:35.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:36:35.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:36:35.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:36:35.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:36:35.014 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:36:35.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:36:35.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:36:35.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:36:35.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:36:35.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:36:35.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:36:35.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:36:35.030 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:36:35.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:36:35.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:36:35.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:36:35.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:36:35.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:36:35.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T11:36:35.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T11:36:35.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T11:36:35.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T11:36:35.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:36:35.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:36:35.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:36:35.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:36:35.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:36:35.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:36:35.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:36:35.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:36:35.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:36:35.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:36:35.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:36:35.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:36:35.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:36:35.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:36:35.039 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:36:35.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:36:35.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:36:35.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:36:35.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:36:35.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:36:35.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:36:35.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:36:35.045 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:36:35.045 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:36:35.045 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:36:35.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:36:35.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:36:35.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:36:35.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T11:36:35.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:36:35.049 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:36:35.049 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:36:35.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T11:36:35.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:36:35.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T11:36:35.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:36:35.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T11:36:35.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_erasure_coded_appends td/osd-scrub-repair 2026-02-17T11:36:35.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:309: TEST_auto_repair_erasure_coded_appends: auto_repair_erasure_coded td/osd-scrub-repair false 2026-02-17T11:36:35.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:268: auto_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T11:36:35.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:269: auto_repair_erasure_coded: local allow_overwrites=false 2026-02-17T11:36:35.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:270: auto_repair_erasure_coded: local poolname=ecpool 2026-02-17T11:36:35.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:273: auto_repair_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-17T11:36:35.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T11:36:35.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T11:36:35.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T11:36:35.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T11:36:35.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T11:36:35.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T11:36:35.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T11:36:35.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:36:35.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:36:35.073 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:36:35.073 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:36:35.073 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:36:35.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:36:35.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T11:36:35.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T11:36:35.101 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T11:36:35.101 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:36:35.101 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:36:35.101 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T11:36:35.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T11:36:35.102 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:36:35.102 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:36:35.102 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:36:35.103 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:36:35.103 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:36:35.103 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:36:35.103 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:36:35.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:36:35.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T11:36:35.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T11:36:35.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:36:35.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:36:35.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T11:36:35.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T11:36:35.160 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:36:35.160 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:36:35.160 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:36:35.161 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:36:35.161 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:36:35.161 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:36:35.161 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:36:35.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:36:35.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T11:36:35.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:274: auto_repair_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-17T11:36:35.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T11:36:35.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T11:36:35.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T11:36:35.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T11:36:35.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T11:36:35.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T11:36:35.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T11:36:35.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:36:35.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:36:35.310 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:36:35.310 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:36:35.310 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:36:35.311 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:36:35.311 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:36:35.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:36:35.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:279: auto_repair_erasure_coded: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:36:35.330 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:280: auto_repair_erasure_coded: seq 0 2 2026-02-17T11:36:35.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:280: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-17T11:36:35.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:281: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:36:35.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:36:35.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:36:35.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T11:36:35.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:36:35.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:36:35.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:36:35.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:36:35.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:36:35.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:36:35.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:36:35.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:36:35.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:36:35.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:36:35.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:36:35.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:36:35.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:36:35.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:36:35.336 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:36:35.336 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:36:35.336 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:36:35.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:36:35.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:36:35.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:36:35.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:36:35.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:36:35.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:36:35.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:36:35.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:36:35.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:36:35.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:36:35.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:36:35.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:36:35.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:36:35.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:36:35.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:36:35.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=907bde1a-4346-4147-926a-afaea655e0db 2026-02-17T11:36:35.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 907bde1a-4346-4147-926a-afaea655e0db' 2026-02-17T11:36:35.338 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 907bde1a-4346-4147-926a-afaea655e0db 2026-02-17T11:36:35.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:36:35.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDDUpRpvgBeFBAAyNtjV4eU/sdhcrPnzW7GFg== 2026-02-17T11:36:35.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDDUpRpvgBeFBAAyNtjV4eU/sdhcrPnzW7GFg=="}' 2026-02-17T11:36:35.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 907bde1a-4346-4147-926a-afaea655e0db -i td/osd-scrub-repair/0/new.json 2026-02-17T11:36:35.437 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:36:35.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T11:36:35.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDDUpRpvgBeFBAAyNtjV4eU/sdhcrPnzW7GFg== --osd-uuid 907bde1a-4346-4147-926a-afaea655e0db 2026-02-17T11:36:35.463 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:35.455+0000 7f6f7d6ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:35.464 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:35.455+0000 7f6f7d6ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:35.465 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:35.459+0000 7f6f7d6ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:35.466 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:35.459+0000 7f6f7d6ba8c0 -1 bdev(0x55c25648dc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:36:35.466 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:35.459+0000 7f6f7d6ba8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T11:36:38.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T11:36:38.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:36:38.259 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:36:38.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T11:36:38.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:36:38.453 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:36:38.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T11:36:38.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:36:38.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:36:38.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:36:38.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:36:38.469 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:38.459+0000 7f569574a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:38.470 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:38.463+0000 7f569574a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:38.471 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:38.463+0000 7f569574a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:38.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T11:36:38.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:36:38.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:36:38.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:36:38.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:36:38.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:38.610 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:36:38.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:36:38.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:38.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:36:38.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:39.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:39.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:39.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:36:39.768 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:36:39.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:39.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:36:39.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:40.163 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:40.155+0000 7f569574a8c0 -1 Falling back to public interface 2026-02-17T11:36:40.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:40.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:40.922 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:36:40.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:36:40.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:40.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:36:41.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:42.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:42.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:42.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:36:42.074 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:36:42.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:42.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:36:42.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:43.144 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:43.135+0000 7f569574a8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:36:43.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:43.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:43.224 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:36:43.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:36:43.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:43.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:36:43.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:44.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:44.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:44.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:36:44.378 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:36:44.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:44.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:36:44.532 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/755253515,v1:127.0.0.1:6803/755253515] [v2:127.0.0.1:6804/755253515,v1:127.0.0.1:6805/755253515] exists,up 907bde1a-4346-4147-926a-afaea655e0db 2026-02-17T11:36:44.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:36:44.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:36:44.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:36:44.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:280: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-17T11:36:44.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:281: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:36:44.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:36:44.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:36:44.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T11:36:44.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:36:44.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:36:44.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:36:44.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:36:44.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:36:44.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:36:44.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:36:44.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:36:44.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:36:44.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:36:44.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:36:44.537 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:36:44.537 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:36:44.537 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:36:44.537 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:36:44.537 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:36:44.537 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:36:44.538 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:36:44.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:36:44.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:36:44.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:36:44.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:36:44.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:36:44.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:36:44.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:36:44.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:36:44.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:36:44.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:36:44.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:36:44.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:36:44.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:36:44.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:36:44.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=9e024f1c-c27f-4249-af9e-df446ec3dd5d 2026-02-17T11:36:44.539 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 9e024f1c-c27f-4249-af9e-df446ec3dd5d 2026-02-17T11:36:44.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 9e024f1c-c27f-4249-af9e-df446ec3dd5d' 2026-02-17T11:36:44.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:36:44.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDMUpRpt0BoIBAAxh130ISLxathc3UQKI0Nlg== 2026-02-17T11:36:44.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDMUpRpt0BoIBAAxh130ISLxathc3UQKI0Nlg=="}' 2026-02-17T11:36:44.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 9e024f1c-c27f-4249-af9e-df446ec3dd5d -i td/osd-scrub-repair/1/new.json 2026-02-17T11:36:44.702 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:36:44.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T11:36:44.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDMUpRpt0BoIBAAxh130ISLxathc3UQKI0Nlg== --osd-uuid 9e024f1c-c27f-4249-af9e-df446ec3dd5d 2026-02-17T11:36:44.730 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:44.723+0000 7f514ee8c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:44.731 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:44.723+0000 7f514ee8c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:44.732 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:44.723+0000 7f514ee8c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:44.733 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:44.723+0000 7f514ee8c8c0 -1 bdev(0x55b74bc0bc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:36:44.733 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:44.723+0000 7f514ee8c8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T11:36:46.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T11:36:46.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:36:46.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T11:36:46.995 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:36:46.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:36:47.184 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:36:47.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T11:36:47.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:36:47.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:36:47.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:36:47.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:36:47.211 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:47.203+0000 7fde482ee8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:47.212 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:47.203+0000 7fde482ee8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:47.213 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:47.207+0000 7fde482ee8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:47.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T11:36:47.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:36:47.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:36:47.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:36:47.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:36:47.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:47.344 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:36:47.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:36:47.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:47.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:36:47.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:48.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:48.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:48.497 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:36:48.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:36:48.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:48.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:36:48.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:48.702 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:48.695+0000 7fde482ee8c0 -1 Falling back to public interface 2026-02-17T11:36:49.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:49.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:49.647 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:36:49.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:36:49.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:49.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:36:49.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:50.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:50.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:50.797 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:36:50.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:36:50.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:50.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:36:50.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:51.732 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:51.723+0000 7fde482ee8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:36:51.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:51.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:51.948 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:36:51.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:36:51.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:51.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:36:52.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:53.099 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:36:53.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:36:53.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:53.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:36:53.249 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1165460113,v1:127.0.0.1:6811/1165460113] [v2:127.0.0.1:6812/1165460113,v1:127.0.0.1:6813/1165460113] exists,up 9e024f1c-c27f-4249-af9e-df446ec3dd5d 2026-02-17T11:36:53.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:36:53.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:36:53.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:36:53.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:280: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-17T11:36:53.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:281: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:36:53.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:36:53.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:36:53.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T11:36:53.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:36:53.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:36:53.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:36:53.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:36:53.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:36:53.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:36:53.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:36:53.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:36:53.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:36:53.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:36:53.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:36:53.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:36:53.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:36:53.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:36:53.254 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:36:53.254 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:36:53.254 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:36:53.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:36:53.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:36:53.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:36:53.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:36:53.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:36:53.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:36:53.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:36:53.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:36:53.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:36:53.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:36:53.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:36:53.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:36:53.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:36:53.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:36:53.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:36:53.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=f179bbd2-3123-4760-8183-51de0b0af584 2026-02-17T11:36:53.257 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 f179bbd2-3123-4760-8183-51de0b0af584 2026-02-17T11:36:53.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 f179bbd2-3123-4760-8183-51de0b0af584' 2026-02-17T11:36:53.257 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:36:53.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDVUpRpCGWtDxAAhMZg1EptmIGQqDLTc9E0GA== 2026-02-17T11:36:53.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDVUpRpCGWtDxAAhMZg1EptmIGQqDLTc9E0GA=="}' 2026-02-17T11:36:53.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new f179bbd2-3123-4760-8183-51de0b0af584 -i td/osd-scrub-repair/2/new.json 2026-02-17T11:36:53.419 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:36:53.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T11:36:53.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQDVUpRpCGWtDxAAhMZg1EptmIGQqDLTc9E0GA== --osd-uuid f179bbd2-3123-4760-8183-51de0b0af584 2026-02-17T11:36:53.454 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:53.446+0000 7f16c69ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:53.456 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:53.446+0000 7f16c69ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:53.457 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:53.450+0000 7f16c69ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:53.457 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:53.450+0000 7f16c69ba8c0 -1 bdev(0x562af5e75c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T11:36:53.458 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:53.450+0000 7f16c69ba8c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T11:36:55.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T11:36:55.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:36:55.724 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T11:36:55.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T11:36:55.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:36:55.913 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T11:36:55.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T11:36:55.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:36:55.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:36:55.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:36:55.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:36:55.930 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:55.922+0000 7f021d35e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:55.931 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:55.922+0000 7f021d35e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:55.932 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:55.922+0000 7f021d35e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:36:56.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T11:36:56.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:36:56.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:36:56.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:36:56.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:36:56.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:56.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:36:56.073 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:36:56.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:56.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:36:56.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:57.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:57.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:57.226 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:36:57.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:36:57.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:57.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:36:57.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:57.431 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:36:57.422+0000 7f021d35e8c0 -1 Falling back to public interface 2026-02-17T11:36:58.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:58.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:58.376 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:36:58.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:36:58.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:58.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:36:58.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:36:59.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:36:59.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:36:59.527 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:36:59.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:36:59.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:36:59.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:36:59.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:00.448 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:00.438+0000 7f021d35e8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T11:37:00.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:37:00.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:00.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:37:00.675 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:37:00.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:00.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:37:00.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:01.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:37:01.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:01.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:37:01.832 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:37:01.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:01.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:37:01.989 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1992125988,v1:127.0.0.1:6819/1992125988] [v2:127.0.0.1:6820/1992125988,v1:127.0.0.1:6821/1992125988] exists,up f179bbd2-3123-4760-8183-51de0b0af584 2026-02-17T11:37:01.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:37:01.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:37:01.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:37:01.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:283: auto_repair_erasure_coded: create_rbd_pool 2026-02-17T11:37:01.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T11:37:02.134 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' does not exist 2026-02-17T11:37:02.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T11:37:02.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T11:37:02.355 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T11:37:02.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:37:03.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T11:37:03.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:284: auto_repair_erasure_coded: wait_for_clean 2026-02-17T11:37:03.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:37:03.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:37:03.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:37:03.649 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:37:03.650 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:37:03.650 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:37:03.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:37:03.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:37:03.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:37:03.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:37:03.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:37:03.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:37:03.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:37:03.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:37:03.757 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:37:03.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:37:03.911 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:37:03.912 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:37:03.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:37:03.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:37:03.912 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:37:03.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T11:37:03.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T11:37:03.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T11:37:03.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:37:03.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:37:04.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T11:37:04.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T11:37:04.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-17T11:37:04.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:37:04.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:37:04.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542146 2026-02-17T11:37:04.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542146 2026-02-17T11:37:04.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-60129542146' 2026-02-17T11:37:04.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:37:04.128 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T11:37:04.128 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:37:04.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:37:04.129 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T11:37:04.129 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:37:04.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T11:37:04.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T11:37:04.131 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836485 2026-02-17T11:37:04.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:37:04.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T11:37:04.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:37:05.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:37:05.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:37:05.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T11:37:05.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:37:06.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:37:06.448 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:37:06.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T11:37:06.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:37:06.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T11:37:06.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:37:06.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:37:06.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T11:37:06.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:37:06.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T11:37:06.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T11:37:06.610 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672964 2026-02-17T11:37:06.610 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:37:06.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T11:37:06.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:37:06.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542146 2026-02-17T11:37:06.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:37:06.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:37:06.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542146 2026-02-17T11:37:06.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:37:06.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542146 2026-02-17T11:37:06.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-17T11:37:06.772 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542146 2026-02-17T11:37:06.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:37:06.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542146 -lt 60129542146 2026-02-17T11:37:06.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:37:06.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:37:06.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:37:07.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T11:37:07.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:37:07.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:37:07.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:37:07.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:37:07.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:37:07.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:37:07.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:37:07.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T11:37:07.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:37:07.276 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:37:07.276 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:37:07.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T11:37:07.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:37:07.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:37:07.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:287: auto_repair_erasure_coded: create_ec_pool ecpool false k=2 m=1 2026-02-17T11:37:07.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T11:37:07.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T11:37:07.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=false 2026-02-17T11:37:07.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T11:37:07.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-17T11:37:07.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T11:37:07.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T11:37:07.967 INFO:tasks.workunit.client.0.trial128.stderr:pool 'ecpool' created 2026-02-17T11:37:07.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:37:08.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' false = true ']' 2026-02-17T11:37:08.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T11:37:08.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:37:08.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:37:08.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:37:08.980 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:37:08.981 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:37:08.981 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:37:08.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:37:08.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:37:08.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:37:09.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:37:09.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:37:09.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:37:09.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:37:09.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:37:09.087 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:37:09.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:37:09.242 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:37:09.242 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:37:09.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:37:09.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:37:09.242 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:37:09.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836488 2026-02-17T11:37:09.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836488 2026-02-17T11:37:09.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488' 2026-02-17T11:37:09.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:37:09.316 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:37:09.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672966 2026-02-17T11:37:09.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672966 2026-02-17T11:37:09.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-42949672966' 2026-02-17T11:37:09.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:37:09.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:37:09.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542148 2026-02-17T11:37:09.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542148 2026-02-17T11:37:09.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-42949672966 2-60129542148' 2026-02-17T11:37:09.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:37:09.461 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836488 2026-02-17T11:37:09.461 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:37:09.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:37:09.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836488 2026-02-17T11:37:09.463 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:37:09.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836488 2026-02-17T11:37:09.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-17T11:37:09.465 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836488 2026-02-17T11:37:09.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:37:09.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836488 2026-02-17T11:37:09.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:37:10.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:37:10.629 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:37:10.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-17T11:37:10.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:37:10.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672966 2026-02-17T11:37:10.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:37:10.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:37:10.781 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672966 2026-02-17T11:37:10.781 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:37:10.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672966 2026-02-17T11:37:10.782 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672966 2026-02-17T11:37:10.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2026-02-17T11:37:10.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:37:10.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672966 -lt 42949672966 2026-02-17T11:37:10.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:37:10.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542148 2026-02-17T11:37:10.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:37:10.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:37:10.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542148 2026-02-17T11:37:10.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:37:10.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542148 2026-02-17T11:37:10.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-17T11:37:10.949 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542148 2026-02-17T11:37:10.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:37:11.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-17T11:37:11.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:37:11.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:37:11.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:37:11.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:37:11.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:37:11.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:37:11.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:37:11.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:37:11.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:37:11.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:37:11.292 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:37:11.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:37:11.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:37:11.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:37:11.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:37:11.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:37:11.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:37:11.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:37:11.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T11:37:11.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:290: auto_repair_erasure_coded: local payload=ABCDEF 2026-02-17T11:37:11.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:291: auto_repair_erasure_coded: echo ABCDEF 2026-02-17T11:37:11.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:292: auto_repair_erasure_coded: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T11:37:11.635 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:296: auto_repair_erasure_coded: get_not_primary ecpool SOMETHING 2026-02-17T11:37:11.635 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=ecpool 2026-02-17T11:37:11.635 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:37:11.636 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary ecpool SOMETHING 2026-02-17T11:37:11.636 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T11:37:11.636 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:37:11.636 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:37:11.636 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:37:11.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=2 2026-02-17T11:37:11.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:37:11.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 2)) | .[0]' 2026-02-17T11:37:11.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:296: auto_repair_erasure_coded: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:37:11.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:37:11.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:37:11.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:37:11.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:37:11.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:37:11.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:37:11.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:37:11.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:37:11.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:37:11.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:37:11.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:37:11.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:37:11.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:37:11.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:37:11.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:37:13.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:37:13.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:37:13.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:37:13.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:37:13.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:37:13.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:37:13.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:37:13.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T11:37:13.899 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::scrub_1.3:head#, (61) No data available 2026-02-17T11:37:13.899 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#-3:00000000:::scrub_1.0:head#, (61) No data available 2026-02-17T11:37:13.899 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T11:37:14.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:37:14.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:37:14.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:37:14.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:37:14.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:37:14.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:37:14.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:37:14.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:37:14.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:37:14.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:37:14.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:37:14.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:37:14.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:37:14.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:37:14.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:37:14.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:37:14.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:37:14.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:37:14.447 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:37:14.447 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:37:14.447 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:37:14.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:37:14.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:37:14.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:37:14.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:37:14.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:37:14.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:37:14.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:37:14.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:37:14.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:37:14.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:37:14.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:37:14.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:37:14.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:37:14.450 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:37:14.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:37:14.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:37:14.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:37:14.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:37:14.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:37:14.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:37:14.470 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:14.462+0000 7f27c958f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:14.472 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:14.462+0000 7f27c958f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:14.473 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:14.462+0000 7f27c958f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:14.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:37:14.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:37:14.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:37:14.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:37:14.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:37:14.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:14.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:37:14.599 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:37:14.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:14.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:37:14.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:15.711 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:15.702+0000 7f27c958f8c0 -1 Falling back to public interface 2026-02-17T11:37:15.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:37:15.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:15.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:37:15.756 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:37:15.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:15.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:37:15.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:16.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:37:16.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:16.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:37:16.909 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:37:16.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:16.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:37:17.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:18.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:37:18.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:18.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:37:18.065 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:37:18.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:18.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:37:18.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:18.674 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:18.666+0000 7f27c958f8c0 -1 osd.1 27 log_to_monitors true 2026-02-17T11:37:19.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:37:19.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:19.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:37:19.222 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:37:19.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:19.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:37:19.376 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 31 up_thru 31 down_at 28 last_clean_interval [10,27) [v2:127.0.0.1:6810/2330179559,v1:127.0.0.1:6811/2330179559] [v2:127.0.0.1:6812/2330179559,v1:127.0.0.1:6813/2330179559] exists,up 9e024f1c-c27f-4249-af9e-df446ec3dd5d 2026-02-17T11:37:19.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:37:19.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:37:19.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:37:19.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:37:19.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:37:19.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:37:19.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:37:19.377 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:37:19.378 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:37:19.378 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:37:19.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:37:19.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:37:19.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:37:19.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:37:19.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:37:19.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:37:19.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:37:19.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:37:19.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:37:19.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:37:19.646 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:37:19.646 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:37:19.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:37:19.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:37:19.647 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:37:19.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836491 2026-02-17T11:37:19.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836491 2026-02-17T11:37:19.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491' 2026-02-17T11:37:19.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:37:19.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:37:19.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986178 2026-02-17T11:37:19.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986178 2026-02-17T11:37:19.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-133143986178' 2026-02-17T11:37:19.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:37:19.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:37:19.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542151 2026-02-17T11:37:19.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542151 2026-02-17T11:37:19.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-133143986178 2-60129542151' 2026-02-17T11:37:19.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:37:19.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836491 2026-02-17T11:37:19.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:37:19.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:37:19.861 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836491 2026-02-17T11:37:19.861 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:37:19.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836491 2026-02-17T11:37:19.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-17T11:37:19.863 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836491 2026-02-17T11:37:19.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:37:20.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-17T11:37:20.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:37:21.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:37:21.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:37:21.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-17T11:37:21.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:37:22.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:37:22.183 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:37:22.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-17T11:37:22.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:37:22.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-133143986178 2026-02-17T11:37:22.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:37:22.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:37:22.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-133143986178 2026-02-17T11:37:22.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:37:22.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986178 2026-02-17T11:37:22.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 133143986178' 2026-02-17T11:37:22.341 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 133143986178 2026-02-17T11:37:22.342 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:37:22.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986178 -lt 133143986178 2026-02-17T11:37:22.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:37:22.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542151 2026-02-17T11:37:22.492 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:37:22.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:37:22.494 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542151 2026-02-17T11:37:22.494 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:37:22.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542151 2026-02-17T11:37:22.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542151' 2026-02-17T11:37:22.496 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542151 2026-02-17T11:37:22.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:37:22.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542152 -lt 60129542151 2026-02-17T11:37:22.659 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:37:22.659 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:37:22.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:37:22.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:37:22.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:37:22.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:37:22.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:37:22.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:37:22.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:37:22.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:37:22.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:37:22.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:37:22.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:37:23.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:37:23.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:37:23.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:37:23.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:37:23.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:37:23.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: auto_repair_erasure_coded: get_pg ecpool SOMETHING 2026-02-17T11:37:23.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:37:23.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:37:23.183 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:37:23.183 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:37:23.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: auto_repair_erasure_coded: local pgid=2.0 2026-02-17T11:37:23.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:299: auto_repair_erasure_coded: get_last_scrub_stamp 2.0 2026-02-17T11:37:23.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:37:23.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:37:23.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:37:23.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:37:23.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:299: auto_repair_erasure_coded: wait_for_scrub 2.0 2026-02-17T11:37:20.577795+0000 2026-02-17T11:37:23.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:37:23.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:37:20.577795+0000 2026-02-17T11:37:23.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:37:23.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:37:23.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:37:23.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:37:23.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:37:23.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:37:23.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:37:23.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:37:23.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:37:20.577795+0000 '>' 2026-02-17T11:37:20.577795+0000 2026-02-17T11:37:23.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:37:24.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:37:24.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:37:24.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:37:24.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:37:24.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:37:24.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:37:24.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:37:24.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:37:20.577795+0000 '>' 2026-02-17T11:37:20.577795+0000 2026-02-17T11:37:24.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:37:25.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:37:25.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:37:25.784 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:37:25.784 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:37:25.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:37:25.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:37:25.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:37:25.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:37:20.577795+0000 '>' 2026-02-17T11:37:20.577795+0000 2026-02-17T11:37:25.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:37:26.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:37:26.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:37:26.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:37:26.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:37:26.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:37:26.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:37:26.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:37:27.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:37:20.577795+0000 '>' 2026-02-17T11:37:20.577795+0000 2026-02-17T11:37:27.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:37:28.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:37:28.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:37:28.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:37:28.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:37:28.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:37:28.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:37:28.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:37:28.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:37:20.577795+0000 '>' 2026-02-17T11:37:20.577795+0000 2026-02-17T11:37:28.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:37:29.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:37:29.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:37:29.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:37:29.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:37:29.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:37:29.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:37:29.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:37:29.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:37:20.577795+0000 '>' 2026-02-17T11:37:20.577795+0000 2026-02-17T11:37:29.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:37:30.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:37:30.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:37:30.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:37:30.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:37:30.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:37:30.404 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:37:30.404 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:37:30.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:37:20.577795+0000 '>' 2026-02-17T11:37:20.577795+0000 2026-02-17T11:37:30.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:37:31.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:37:31.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:37:31.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:37:31.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:37:31.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:37:31.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:37:31.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:37:31.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:37:26.628409+0000 '>' 2026-02-17T11:37:20.577795+0000 2026-02-17T11:37:31.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:37:31.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:300: auto_repair_erasure_coded: wait_for_clean 2026-02-17T11:37:31.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:37:31.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:37:31.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:37:31.714 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:37:31.714 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:37:31.714 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:37:31.715 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:37:31.715 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:37:31.715 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:37:31.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:37:31.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:37:31.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:37:31.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:37:31.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:37:31.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:37:31.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:37:31.974 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:37:31.975 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:37:31.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:37:31.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:37:31.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:37:32.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836494 2026-02-17T11:37:32.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836494 2026-02-17T11:37:32.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836494' 2026-02-17T11:37:32.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:37:32.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:37:32.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986181 2026-02-17T11:37:32.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986181 2026-02-17T11:37:32.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836494 1-133143986181' 2026-02-17T11:37:32.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:37:32.121 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:37:32.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542155 2026-02-17T11:37:32.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542155 2026-02-17T11:37:32.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836494 1-133143986181 2-60129542155' 2026-02-17T11:37:32.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:37:32.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836494 2026-02-17T11:37:32.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:37:32.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:37:32.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836494 2026-02-17T11:37:32.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:37:32.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836494 2026-02-17T11:37:32.197 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836494 2026-02-17T11:37:32.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836494' 2026-02-17T11:37:32.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:37:32.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836494 2026-02-17T11:37:32.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:37:33.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:37:33.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:37:33.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836494 2026-02-17T11:37:33.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:37:34.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:37:34.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:37:34.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836494 -lt 21474836494 2026-02-17T11:37:34.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:37:34.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-133143986181 2026-02-17T11:37:34.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:37:34.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:37:34.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-133143986181 2026-02-17T11:37:34.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:37:34.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986181 2026-02-17T11:37:34.678 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 133143986181 2026-02-17T11:37:34.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 133143986181' 2026-02-17T11:37:34.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:37:34.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986181 -lt 133143986181 2026-02-17T11:37:34.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:37:34.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542155 2026-02-17T11:37:34.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:37:34.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:37:34.832 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542155 2026-02-17T11:37:34.832 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:37:34.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542155 2026-02-17T11:37:34.834 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542155 2026-02-17T11:37:34.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542155' 2026-02-17T11:37:34.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:37:34.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542155 -lt 60129542155 2026-02-17T11:37:34.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:37:34.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:37:34.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:37:35.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:37:35.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:37:35.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:37:35.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:37:35.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:37:35.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:37:35.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:37:35.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:37:35.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:37:35.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:37:35.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:37:35.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:37:35.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:37:35.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:37:35.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:37:35.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: auto_repair_erasure_coded: get_not_primary ecpool SOMETHING 2026-02-17T11:37:35.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=ecpool 2026-02-17T11:37:35.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:37:35.515 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary ecpool SOMETHING 2026-02-17T11:37:35.515 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T11:37:35.515 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:37:35.516 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:37:35.516 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:37:35.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=2 2026-02-17T11:37:35.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:37:35.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 2)) | .[0]' 2026-02-17T11:37:35.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: auto_repair_erasure_coded: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:37:35.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:37:35.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:37:35.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:37:35.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:37:35.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:37:35.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:37:35.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:37:35.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:37:35.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:37:35.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:37:35.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:37:35.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:37:35.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:37:35.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:37:35.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:37:37.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:37:37.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:37:37.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:37:37.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:37:37.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:37:37.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:37:37.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:37:37.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T11:37:37.441 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::scrub_1.3:head#, (61) No data available 2026-02-17T11:37:37.441 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#-3:00000000:::scrub_1.0:head#, (61) No data available 2026-02-17T11:37:37.442 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:37:37.442 INFO:tasks.workunit.client.0.trial128.stdout:hinfo_key 2026-02-17T11:37:37.442 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:37:37.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:37:37.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:37:37.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:37:37.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:37:37.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:37:37.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:37:37.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:37:37.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:37:37.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:37:37.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:37:37.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:37:37.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:37:37.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:37:37.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:37:37.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:37:37.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:37:37.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:37:37.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:37:37.977 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:37:37.977 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:37:37.977 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:37:37.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:37:37.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:37:37.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:37:37.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:37:37.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:37:37.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:37:37.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:37:37.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:37:37.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:37:37.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:37:37.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:37:37.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:37:37.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:37:37.978 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:37:37.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:37:37.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:37:37.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:37:37.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:37:37.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:37:37.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:37:37.999 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:37.993+0000 7f64b654a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:38.000 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:37.993+0000 7f64b654a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:38.001 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:37.997+0000 7f64b654a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:38.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:37:38.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:37:38.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:37:38.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:37:38.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:37:38.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:38.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:37:38.134 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:37:38.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:38.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:37:38.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:39.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:37:39.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:39.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:37:39.290 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:37:39.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:39.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:37:39.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:39.455 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:39.449+0000 7f64b654a8c0 -1 Falling back to public interface 2026-02-17T11:37:40.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:37:40.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:40.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:37:40.441 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:37:40.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:40.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:37:40.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:41.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:37:41.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:41.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:37:41.592 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:37:41.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:41.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:37:41.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:42.479 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:42.473+0000 7f64b654a8c0 -1 osd.1 33 log_to_monitors true 2026-02-17T11:37:42.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:37:42.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:42.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:37:42.741 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:37:42.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:42.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:37:42.893 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 37 up_thru 37 down_at 34 last_clean_interval [31,33) [v2:127.0.0.1:6810/3393133879,v1:127.0.0.1:6811/3393133879] [v2:127.0.0.1:6812/3393133879,v1:127.0.0.1:6813/3393133879] exists,up 9e024f1c-c27f-4249-af9e-df446ec3dd5d 2026-02-17T11:37:42.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:37:42.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:37:42.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:37:42.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:37:42.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:37:42.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:37:42.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:37:42.893 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:37:42.894 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:37:42.894 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:37:42.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:37:42.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:37:42.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:37:43.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:37:43.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:37:43.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:37:43.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:37:43.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:37:43.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:37:43.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:37:43.160 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:37:43.160 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:37:43.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:37:43.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:37:43.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:37:43.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836497 2026-02-17T11:37:43.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836497 2026-02-17T11:37:43.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497' 2026-02-17T11:37:43.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:37:43.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:37:43.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789954 2026-02-17T11:37:43.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789954 2026-02-17T11:37:43.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-158913789954' 2026-02-17T11:37:43.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:37:43.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:37:43.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542158 2026-02-17T11:37:43.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542158 2026-02-17T11:37:43.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-158913789954 2-60129542158' 2026-02-17T11:37:43.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:37:43.379 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836497 2026-02-17T11:37:43.380 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:37:43.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:37:43.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836497 2026-02-17T11:37:43.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:37:43.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836497 2026-02-17T11:37:43.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2026-02-17T11:37:43.383 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836497 2026-02-17T11:37:43.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:37:43.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836497 2026-02-17T11:37:43.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:37:44.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:37:44.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:37:44.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836497 -lt 21474836497 2026-02-17T11:37:44.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:37:44.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-158913789954 2026-02-17T11:37:44.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:37:44.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:37:44.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-158913789954 2026-02-17T11:37:44.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:37:44.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789954 2026-02-17T11:37:44.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 158913789954' 2026-02-17T11:37:44.704 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 158913789954 2026-02-17T11:37:44.705 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:37:44.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789954 -lt 158913789954 2026-02-17T11:37:44.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:37:44.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542158 2026-02-17T11:37:44.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:37:44.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:37:44.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542158 2026-02-17T11:37:44.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:37:44.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542158 2026-02-17T11:37:44.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542158' 2026-02-17T11:37:44.871 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542158 2026-02-17T11:37:44.871 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:37:45.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542158 -lt 60129542158 2026-02-17T11:37:45.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:37:45.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:37:45.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:37:45.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:37:45.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:37:45.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:37:45.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:37:45.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:37:45.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:37:45.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:37:45.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:37:45.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:37:45.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:37:45.361 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:37:45.361 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:37:45.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:37:45.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:37:45.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:37:45.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:304: auto_repair_erasure_coded: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:37:45.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:305: auto_repair_erasure_coded: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:37:45.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T11:37:45.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:37:45.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:37:45.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:37:45.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:37:45.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:37:45.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:37:45.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:37:45.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:37:45.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:37:45.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:37:45.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:37:45.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:37:45.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:37:45.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:37:45.672 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:37:45.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:37:45.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:37:45.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:37:45.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:37:45.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:37:45.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:37:45.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:37:45.691 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:37:45.691 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:37:45.691 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:37:45.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:37:45.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:37:45.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:37:45.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T11:37:45.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T11:37:45.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T11:37:45.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T11:37:45.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:37:45.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:37:45.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:37:45.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:37:45.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:37:45.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:37:45.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:37:45.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:37:45.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:37:45.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:37:45.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:37:45.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:37:45.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:37:45.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:37:45.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:37:45.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:37:45.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:37:45.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:37:45.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:37:45.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:37:45.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:37:45.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:37:45.705 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:37:45.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:37:45.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:37:45.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:37:45.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:37:45.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:37:45.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T11:37:45.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:37:45.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:37:45.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:37:45.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T11:37:45.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:37:45.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T11:37:45.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:37:45.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T11:37:45.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_auto_repair_erasure_coded_overwrites td/osd-scrub-repair 2026-02-17T11:37:45.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:313: TEST_auto_repair_erasure_coded_overwrites: '[' true = true ']' 2026-02-17T11:37:45.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:314: TEST_auto_repair_erasure_coded_overwrites: auto_repair_erasure_coded td/osd-scrub-repair true 2026-02-17T11:37:45.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:268: auto_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T11:37:45.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:269: auto_repair_erasure_coded: local allow_overwrites=true 2026-02-17T11:37:45.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:270: auto_repair_erasure_coded: local poolname=ecpool 2026-02-17T11:37:45.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:273: auto_repair_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-17T11:37:45.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T11:37:45.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T11:37:45.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T11:37:45.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T11:37:45.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T11:37:45.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T11:37:45.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T11:37:45.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:37:45.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:37:45.733 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:37:45.733 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:37:45.733 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:37:45.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:37:45.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T11:37:45.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T11:37:45.764 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T11:37:45.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:37:45.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:37:45.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T11:37:45.766 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T11:37:45.766 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:37:45.766 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:37:45.766 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:37:45.766 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:37:45.767 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:37:45.767 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:37:45.767 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:37:45.767 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:37:45.767 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T11:37:45.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T11:37:45.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:37:45.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:37:45.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T11:37:45.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T11:37:45.827 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:37:45.827 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:37:45.828 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:37:45.828 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:37:45.828 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:37:45.828 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:37:45.829 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:37:45.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:37:45.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T11:37:45.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:274: auto_repair_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-17T11:37:45.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T11:37:45.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T11:37:45.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T11:37:45.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T11:37:45.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T11:37:45.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T11:37:45.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T11:37:45.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:37:45.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:37:45.968 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:37:45.969 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:37:45.969 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:37:45.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:37:45.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:37:45.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:37:45.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:279: auto_repair_erasure_coded: local 'ceph_osd_args=--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:37:45.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:280: auto_repair_erasure_coded: seq 0 2 2026-02-17T11:37:45.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:280: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-17T11:37:45.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:281: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 0 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:37:45.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:37:45.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:37:45.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T11:37:45.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:37:45.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:37:45.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:37:45.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:37:45.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:37:45.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:37:45.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:37:45.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:37:45.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:37:45.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:37:45.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:37:45.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:37:45.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:37:45.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:37:45.994 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:37:45.994 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:37:45.994 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:37:45.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:37:45.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:37:45.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:37:45.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:37:45.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:37:45.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:37:45.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:37:45.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:37:45.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:37:45.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:37:45.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:37:45.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:37:45.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:37:45.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:37:45.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:37:45.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3ac8e183-64ed-4762-8de2-6f88bbad68e6 2026-02-17T11:37:45.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 3ac8e183-64ed-4762-8de2-6f88bbad68e6' 2026-02-17T11:37:45.996 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 3ac8e183-64ed-4762-8de2-6f88bbad68e6 2026-02-17T11:37:45.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:37:46.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAKU5RpdQ0TABAAwILJk/2z7DyabB7Wv1KMuA== 2026-02-17T11:37:46.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAKU5RpdQ0TABAAwILJk/2z7DyabB7Wv1KMuA=="}' 2026-02-17T11:37:46.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3ac8e183-64ed-4762-8de2-6f88bbad68e6 -i td/osd-scrub-repair/0/new.json 2026-02-17T11:37:46.097 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:37:46.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T11:37:46.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQAKU5RpdQ0TABAAwILJk/2z7DyabB7Wv1KMuA== --osd-uuid 3ac8e183-64ed-4762-8de2-6f88bbad68e6 2026-02-17T11:37:46.124 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:46.117+0000 7f4aa831a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:46.125 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:46.117+0000 7f4aa831a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:46.126 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:46.121+0000 7f4aa831a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:46.126 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:46.121+0000 7f4aa831a8c0 -1 bdev(0x55c955659c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:37:46.127 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:46.121+0000 7f4aa831a8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T11:37:48.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T11:37:48.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:37:48.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T11:37:48.688 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:37:48.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:37:48.878 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:37:48.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T11:37:48.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:37:48.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:37:48.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:37:48.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:37:48.904 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:48.897+0000 7f904b7f48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:48.905 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:48.897+0000 7f904b7f48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:48.906 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:48.901+0000 7f904b7f48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:49.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T11:37:49.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:37:49.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:37:49.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:37:49.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:37:49.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:49.036 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:37:49.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:37:49.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:49.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:37:49.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:50.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:37:50.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:50.183 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:37:50.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:37:50.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:50.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:37:50.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:50.407 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:50.401+0000 7f904b7f48c0 -1 Falling back to public interface 2026-02-17T11:37:51.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:37:51.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:51.334 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:37:51.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:37:51.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:51.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:37:51.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:52.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:37:52.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:52.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:37:52.483 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:37:52.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:52.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:37:52.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:53.391 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:53.385+0000 7f904b7f48c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:37:53.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:37:53.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:53.635 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:37:53.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:37:53.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:53.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:37:53.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:54.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:37:54.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:54.786 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:37:54.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:37:54.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:54.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:37:54.935 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2600283912,v1:127.0.0.1:6803/2600283912] [v2:127.0.0.1:6804/2600283912,v1:127.0.0.1:6805/2600283912] exists,up 3ac8e183-64ed-4762-8de2-6f88bbad68e6 2026-02-17T11:37:54.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:37:54.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:37:54.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:37:54.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:280: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-17T11:37:54.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:281: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:37:54.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:37:54.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:37:54.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:37:54.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:37:54.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:37:54.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:37:54.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:37:54.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:37:54.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:37:54.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:37:54.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:37:54.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:37:54.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:37:54.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:37:54.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:37:54.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=f25cefe1-5783-431d-bc58-9f267f54f9f2 2026-02-17T11:37:54.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 f25cefe1-5783-431d-bc58-9f267f54f9f2' 2026-02-17T11:37:54.941 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 f25cefe1-5783-431d-bc58-9f267f54f9f2 2026-02-17T11:37:54.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:37:54.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQASU5RpzBaKOBAALOE6j6KpWmo1JhARAl4xCQ== 2026-02-17T11:37:54.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQASU5RpzBaKOBAALOE6j6KpWmo1JhARAl4xCQ=="}' 2026-02-17T11:37:54.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new f25cefe1-5783-431d-bc58-9f267f54f9f2 -i td/osd-scrub-repair/1/new.json 2026-02-17T11:37:55.108 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:37:55.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T11:37:55.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQASU5RpzBaKOBAALOE6j6KpWmo1JhARAl4xCQ== --osd-uuid f25cefe1-5783-431d-bc58-9f267f54f9f2 2026-02-17T11:37:55.135 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:55.129+0000 7f9bf7bf58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:55.137 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:55.129+0000 7f9bf7bf58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:55.137 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:55.133+0000 7f9bf7bf58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:55.138 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:55.133+0000 7f9bf7bf58c0 -1 bdev(0x55fb336c1c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:37:55.138 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:55.133+0000 7f9bf7bf58c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T11:37:57.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T11:37:57.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:37:57.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T11:37:57.451 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:37:57.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:37:57.640 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:37:57.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T11:37:57.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:37:57.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:37:57.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:37:57.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:37:57.665 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:57.657+0000 7f5ccfb098c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:57.667 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:57.661+0000 7f5ccfb098c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:57.668 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:57.661+0000 7f5ccfb098c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:37:57.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T11:37:57.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:37:57.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:37:57.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:37:57.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:37:57.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:57.796 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:37:57.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:37:57.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:57.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:37:57.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:58.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:37:58.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:37:58.951 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:37:58.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:37:58.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:37:58.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:37:59.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:37:59.372 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:37:59.365+0000 7f5ccfb098c0 -1 Falling back to public interface 2026-02-17T11:38:00.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:00.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:00.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:38:00.103 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:38:00.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:00.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:38:00.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:38:01.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:01.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:01.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:38:01.253 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:38:01.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:01.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:38:01.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:38:02.401 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:02.393+0000 7f5ccfb098c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:38:02.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:02.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:02.408 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:38:02.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:38:02.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:02.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:38:02.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:38:03.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:03.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:03.556 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:38:03.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:38:03.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:03.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:38:03.706 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2668306605,v1:127.0.0.1:6811/2668306605] [v2:127.0.0.1:6812/2668306605,v1:127.0.0.1:6813/2668306605] exists,up f25cefe1-5783-431d-bc58-9f267f54f9f2 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:280: auto_repair_erasure_coded: for id in $(seq 0 2) 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:281: auto_repair_erasure_coded: run_osd td/osd-scrub-repair 2 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:38:03.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:38:03.708 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:38:03.708 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:38:03.708 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:38:03.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:38:03.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:38:03.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:38:03.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:38:03.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:38:03.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:38:03.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:38:03.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:38:03.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:38:03.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:38:03.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:38:03.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:38:03.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:38:03.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:38:03.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:38:03.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=05d29e13-0522-4242-9031-82eec2ff18ff 2026-02-17T11:38:03.710 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 05d29e13-0522-4242-9031-82eec2ff18ff 2026-02-17T11:38:03.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 05d29e13-0522-4242-9031-82eec2ff18ff' 2026-02-17T11:38:03.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:38:03.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAbU5RprCrGKhAA0pUpZTYr60zMU8l+hmZeYw== 2026-02-17T11:38:03.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAbU5RprCrGKhAA0pUpZTYr60zMU8l+hmZeYw=="}' 2026-02-17T11:38:03.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 05d29e13-0522-4242-9031-82eec2ff18ff -i td/osd-scrub-repair/2/new.json 2026-02-17T11:38:03.866 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:38:03.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T11:38:03.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQAbU5RprCrGKhAA0pUpZTYr60zMU8l+hmZeYw== --osd-uuid 05d29e13-0522-4242-9031-82eec2ff18ff 2026-02-17T11:38:03.892 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:03.885+0000 7f9f716c18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:03.894 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:03.889+0000 7f9f716c18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:03.895 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:03.889+0000 7f9f716c18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:03.896 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:03.889+0000 7f9f716c18c0 -1 bdev(0x56465ae23c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T11:38:03.896 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:03.889+0000 7f9f716c18c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T11:38:06.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T11:38:06.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:38:06.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T11:38:06.189 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T11:38:06.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:38:06.380 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T11:38:06.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T11:38:06.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:38:06.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:38:06.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:38:06.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:38:06.405 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:06.397+0000 7f9f711088c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:06.407 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:06.401+0000 7f9f711088c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:06.408 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:06.401+0000 7f9f711088c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:06.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T11:38:06.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:38:06.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:38:06.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:38:06.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:38:06.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:06.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:38:06.534 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:38:06.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:06.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:38:06.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:38:07.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:07.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:07.686 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:38:07.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:38:07.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:07.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:38:07.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:38:07.907 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:07.901+0000 7f9f711088c0 -1 Falling back to public interface 2026-02-17T11:38:08.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:08.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:08.833 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:38:08.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:38:08.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:08.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:38:08.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:38:09.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:09.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:09.984 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:38:09.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:38:09.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:09.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:38:10.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:38:10.897 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:10.888+0000 7f9f711088c0 -1 osd.2 0 log_to_monitors true 2026-02-17T11:38:11.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:11.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:11.135 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:38:11.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:38:11.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:11.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:38:11.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:38:12.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:12.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:12.284 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:38:12.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:38:12.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:12.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:38:12.432 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1345374012,v1:127.0.0.1:6819/1345374012] [v2:127.0.0.1:6820/1345374012,v1:127.0.0.1:6821/1345374012] exists,up 05d29e13-0522-4242-9031-82eec2ff18ff 2026-02-17T11:38:12.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:38:12.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:38:12.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:38:12.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:283: auto_repair_erasure_coded: create_rbd_pool 2026-02-17T11:38:12.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T11:38:12.572 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' does not exist 2026-02-17T11:38:12.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T11:38:12.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T11:38:12.793 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T11:38:12.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:38:13.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T11:38:14.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:284: auto_repair_erasure_coded: wait_for_clean 2026-02-17T11:38:14.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:38:14.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:38:14.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:38:14.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:38:14.087 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:38:14.087 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:38:14.087 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:38:14.087 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:38:14.087 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:38:14.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:38:14.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:38:14.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:38:14.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:38:14.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:38:14.198 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:38:14.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:38:14.350 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:38:14.350 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:38:14.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:38:14.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:38:14.350 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:38:14.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T11:38:14.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T11:38:14.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T11:38:14.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:38:14.421 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:38:14.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705668 2026-02-17T11:38:14.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705668 2026-02-17T11:38:14.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705668' 2026-02-17T11:38:14.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:38:14.495 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:38:14.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542146 2026-02-17T11:38:14.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542146 2026-02-17T11:38:14.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705668 2-60129542146' 2026-02-17T11:38:14.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:38:14.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T11:38:14.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:38:14.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:38:14.566 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T11:38:14.567 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:38:14.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T11:38:14.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T11:38:14.568 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836485 2026-02-17T11:38:14.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:38:14.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T11:38:14.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:38:15.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:38:15.732 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:38:15.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T11:38:15.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:38:16.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:38:16.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:38:17.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T11:38:17.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:38:17.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705668 2026-02-17T11:38:17.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:38:17.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:38:17.043 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705668 2026-02-17T11:38:17.043 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:38:17.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705668 2026-02-17T11:38:17.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-17T11:38:17.045 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705668 2026-02-17T11:38:17.045 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:38:17.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-17T11:38:17.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:38:17.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542146 2026-02-17T11:38:17.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:38:17.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:38:17.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542146 2026-02-17T11:38:17.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:38:17.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542146 2026-02-17T11:38:17.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-17T11:38:17.203 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542146 2026-02-17T11:38:17.203 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:38:17.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542146 -lt 60129542146 2026-02-17T11:38:17.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:38:17.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:38:17.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:38:17.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T11:38:17.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:38:17.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:38:17.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:38:17.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:38:17.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:38:17.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:38:17.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:38:17.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T11:38:17.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:38:17.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:38:17.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:38:17.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T11:38:17.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:38:17.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:38:17.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:287: auto_repair_erasure_coded: create_ec_pool ecpool true k=2 m=1 2026-02-17T11:38:17.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T11:38:17.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T11:38:17.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=true 2026-02-17T11:38:17.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T11:38:17.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-17T11:38:18.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T11:38:18.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T11:38:18.428 INFO:tasks.workunit.client.0.trial128.stderr:pool 'ecpool' created 2026-02-17T11:38:18.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:38:19.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' true = true ']' 2026-02-17T11:38:19.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2508: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-17T11:38:19.638 INFO:tasks.workunit.client.0.trial128.stderr:set pool 2 allow_ec_overwrites to true 2026-02-17T11:38:19.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T11:38:19.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:38:19.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:38:19.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:38:19.649 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:38:19.650 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:38:19.650 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:38:19.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:38:19.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:38:19.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:38:19.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:38:19.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:38:19.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:38:19.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:38:19.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:38:19.766 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:38:19.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:38:19.914 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:38:19.914 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:38:19.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:38:19.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:38:19.914 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:38:19.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836488 2026-02-17T11:38:19.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836488 2026-02-17T11:38:19.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488' 2026-02-17T11:38:19.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:38:19.986 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:38:20.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705670 2026-02-17T11:38:20.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705670 2026-02-17T11:38:20.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-38654705670' 2026-02-17T11:38:20.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:38:20.056 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:38:20.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542148 2026-02-17T11:38:20.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542148 2026-02-17T11:38:20.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-38654705670 2-60129542148' 2026-02-17T11:38:20.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:38:20.128 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836488 2026-02-17T11:38:20.128 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:38:20.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:38:20.130 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836488 2026-02-17T11:38:20.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:38:20.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836488 2026-02-17T11:38:20.132 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836488 2026-02-17T11:38:20.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-17T11:38:20.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:38:20.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836488 2026-02-17T11:38:20.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:38:21.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:38:21.295 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:38:21.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-17T11:38:21.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:38:21.444 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705670 2026-02-17T11:38:21.444 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:38:21.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:38:21.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705670 2026-02-17T11:38:21.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:38:21.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705670 2026-02-17T11:38:21.448 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705670 2026-02-17T11:38:21.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-17T11:38:21.448 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:38:21.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705670 -lt 38654705670 2026-02-17T11:38:21.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:38:21.612 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542148 2026-02-17T11:38:21.612 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:38:21.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:38:21.614 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542148 2026-02-17T11:38:21.614 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:38:21.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542148 2026-02-17T11:38:21.616 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542148 2026-02-17T11:38:21.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-17T11:38:21.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:38:21.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-17T11:38:21.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:38:21.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:38:21.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:38:21.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:38:21.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:38:21.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:38:21.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:38:21.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:38:21.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:38:21.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:38:21.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:38:22.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:38:22.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:38:22.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:38:22.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:38:22.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:38:22.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:38:22.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:38:22.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T11:38:22.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:290: auto_repair_erasure_coded: local payload=ABCDEF 2026-02-17T11:38:22.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:291: auto_repair_erasure_coded: echo ABCDEF 2026-02-17T11:38:22.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:292: auto_repair_erasure_coded: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T11:38:22.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:296: auto_repair_erasure_coded: get_not_primary ecpool SOMETHING 2026-02-17T11:38:22.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=ecpool 2026-02-17T11:38:22.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:38:22.301 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary ecpool SOMETHING 2026-02-17T11:38:22.301 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T11:38:22.301 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:38:22.302 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:38:22.302 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:38:22.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=2 2026-02-17T11:38:22.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:38:22.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 2)) | .[0]' 2026-02-17T11:38:22.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:296: auto_repair_erasure_coded: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:38:22.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:38:22.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:38:22.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:38:22.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:38:22.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:38:22.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:38:22.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:38:22.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:38:22.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:38:22.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:38:22.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:38:22.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:38:22.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:38:22.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:38:22.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:38:23.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:38:23.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:38:23.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:38:23.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:38:23.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:38:23.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:38:23.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:38:23.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T11:38:24.565 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::scrub_1.3:head#, (61) No data available 2026-02-17T11:38:24.565 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#-3:00000000:::scrub_1.0:head#, (61) No data available 2026-02-17T11:38:24.566 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T11:38:25.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:38:25.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:38:25.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:38:25.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:38:25.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:38:25.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:38:25.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:38:25.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:38:25.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:38:25.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:38:25.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:38:25.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:38:25.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:38:25.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:38:25.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:38:25.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:38:25.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:38:25.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:38:25.103 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:38:25.103 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:38:25.103 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:38:25.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:38:25.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:38:25.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:38:25.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:38:25.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:38:25.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:38:25.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:38:25.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:38:25.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:38:25.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:38:25.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:38:25.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:38:25.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:38:25.106 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:38:25.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:38:25.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:38:25.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:38:25.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:38:25.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:38:25.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:38:25.128 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:25.120+0000 7f12ff4f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:25.129 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:25.120+0000 7f12ff4f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:25.130 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:25.124+0000 7f12ff4f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:25.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:38:25.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:38:25.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:38:25.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:38:25.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:38:25.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:25.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:38:25.262 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:38:25.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:25.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:38:25.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:38:26.396 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:26.388+0000 7f12ff4f38c0 -1 Falling back to public interface 2026-02-17T11:38:26.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:26.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:26.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:38:26.414 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:38:26.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:26.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:38:26.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:38:27.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:27.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:27.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:38:27.565 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:38:27.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:27.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:38:27.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:38:28.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:28.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:28.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:38:28.717 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:38:28.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:28.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:38:28.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:38:29.399 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:29.392+0000 7f12ff4f38c0 -1 osd.1 29 log_to_monitors true 2026-02-17T11:38:29.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:29.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:29.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:38:29.867 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:38:29.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:29.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:38:30.019 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 33 up_thru 33 down_at 30 last_clean_interval [9,29) [v2:127.0.0.1:6810/307231121,v1:127.0.0.1:6811/307231121] [v2:127.0.0.1:6812/307231121,v1:127.0.0.1:6813/307231121] exists,up f25cefe1-5783-431d-bc58-9f267f54f9f2 2026-02-17T11:38:30.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:38:30.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:38:30.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:38:30.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:38:30.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:38:30.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:38:30.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:38:30.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:38:30.020 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:38:30.020 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:38:30.021 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:38:30.021 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:38:30.021 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:38:30.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:38:30.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:38:30.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:38:30.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:38:30.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:38:30.132 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:38:30.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:38:30.281 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:38:30.281 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:38:30.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:38:30.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:38:30.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:38:30.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836491 2026-02-17T11:38:30.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836491 2026-02-17T11:38:30.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491' 2026-02-17T11:38:30.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:38:30.353 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:38:30.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=141733920770 2026-02-17T11:38:30.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 141733920770 2026-02-17T11:38:30.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-141733920770' 2026-02-17T11:38:30.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:38:30.424 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:38:30.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542151 2026-02-17T11:38:30.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542151 2026-02-17T11:38:30.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-141733920770 2-60129542151' 2026-02-17T11:38:30.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:38:30.497 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836491 2026-02-17T11:38:30.497 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:38:30.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:38:30.499 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836491 2026-02-17T11:38:30.499 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:38:30.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836491 2026-02-17T11:38:30.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-17T11:38:30.501 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836491 2026-02-17T11:38:30.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:38:30.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-17T11:38:30.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:38:31.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:38:31.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:38:31.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-17T11:38:31.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:38:32.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:38:32.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:38:32.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-17T11:38:32.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:38:32.962 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-141733920770 2026-02-17T11:38:32.962 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:38:32.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:38:32.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-141733920770 2026-02-17T11:38:32.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:38:32.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=141733920770 2026-02-17T11:38:32.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 141733920770' 2026-02-17T11:38:32.965 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 141733920770 2026-02-17T11:38:32.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:38:33.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920770 -lt 141733920770 2026-02-17T11:38:33.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:38:33.129 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542151 2026-02-17T11:38:33.130 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:38:33.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:38:33.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542151 2026-02-17T11:38:33.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:38:33.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542151 2026-02-17T11:38:33.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542151' 2026-02-17T11:38:33.133 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542151 2026-02-17T11:38:33.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:38:33.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542152 -lt 60129542151 2026-02-17T11:38:33.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:38:33.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:38:33.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:38:33.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:38:33.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:38:33.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:38:33.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:38:33.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:38:33.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:38:33.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:38:33.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:38:33.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:38:33.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:38:33.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:38:33.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:38:33.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:38:33.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:38:33.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:38:33.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: auto_repair_erasure_coded: get_pg ecpool SOMETHING 2026-02-17T11:38:33.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:38:33.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:38:33.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:38:33.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:38:33.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:298: auto_repair_erasure_coded: local pgid=2.0 2026-02-17T11:38:33.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:299: auto_repair_erasure_coded: get_last_scrub_stamp 2.0 2026-02-17T11:38:33.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:38:33.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:38:33.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:38:33.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:38:34.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:299: auto_repair_erasure_coded: wait_for_scrub 2.0 2026-02-17T11:38:31.045675+0000 2026-02-17T11:38:34.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:38:34.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:38:31.045675+0000 2026-02-17T11:38:34.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:38:34.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:38:34.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:38:34.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:38:34.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:38:34.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:38:34.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:38:34.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:38:34.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:38:31.045675+0000 '>' 2026-02-17T11:38:31.045675+0000 2026-02-17T11:38:34.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:38:35.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:38:35.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:38:35.245 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:38:35.245 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:38:35.245 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:38:35.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:38:35.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:38:35.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:38:31.045675+0000 '>' 2026-02-17T11:38:31.045675+0000 2026-02-17T11:38:35.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:38:36.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:38:36.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:38:36.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:38:36.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:38:36.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:38:36.396 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:38:36.396 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:38:36.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:38:31.045675+0000 '>' 2026-02-17T11:38:31.045675+0000 2026-02-17T11:38:36.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:38:37.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:38:37.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:38:37.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:38:37.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:38:37.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:38:37.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:38:37.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:38:37.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:38:31.045675+0000 '>' 2026-02-17T11:38:31.045675+0000 2026-02-17T11:38:37.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:38:38.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:38:38.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:38:38.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:38:38.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:38:38.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:38:38.705 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:38:38.705 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:38:38.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:38:31.045675+0000 '>' 2026-02-17T11:38:31.045675+0000 2026-02-17T11:38:38.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:38:39.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:38:39.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:38:39.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:38:39.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:38:39.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:38:39.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:38:39.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:38:40.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:38:31.045675+0000 '>' 2026-02-17T11:38:31.045675+0000 2026-02-17T11:38:40.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:38:41.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:38:41.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:38:41.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:38:41.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:38:41.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:38:41.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:38:41.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:38:41.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:38:31.045675+0000 '>' 2026-02-17T11:38:31.045675+0000 2026-02-17T11:38:41.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:38:42.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:38:42.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:38:42.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:38:42.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:38:42.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:38:42.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:38:42.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:38:42.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:38:31.045675+0000 '>' 2026-02-17T11:38:31.045675+0000 2026-02-17T11:38:42.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:38:43.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:38:43.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:38:43.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:38:43.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:38:43.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:38:43.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:38:43.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:38:43.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:38:37.941214+0000 '>' 2026-02-17T11:38:31.045675+0000 2026-02-17T11:38:43.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:38:43.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:300: auto_repair_erasure_coded: wait_for_clean 2026-02-17T11:38:43.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:38:43.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:38:43.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:38:43.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:38:43.466 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:38:43.466 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:38:43.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:38:43.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:38:43.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:38:43.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:38:43.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:38:43.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:38:43.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:38:43.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:38:43.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:38:43.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:38:43.735 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:38:43.735 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:38:43.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:38:43.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:38:43.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:38:43.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836494 2026-02-17T11:38:43.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836494 2026-02-17T11:38:43.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836494' 2026-02-17T11:38:43.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:38:43.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:38:43.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=141733920773 2026-02-17T11:38:43.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 141733920773 2026-02-17T11:38:43.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836494 1-141733920773' 2026-02-17T11:38:43.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:38:43.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:38:43.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542155 2026-02-17T11:38:43.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542155 2026-02-17T11:38:43.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836494 1-141733920773 2-60129542155' 2026-02-17T11:38:43.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:38:43.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836494 2026-02-17T11:38:43.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:38:43.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:38:43.961 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836494 2026-02-17T11:38:43.961 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:38:43.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836494 2026-02-17T11:38:43.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836494' 2026-02-17T11:38:43.963 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836494 2026-02-17T11:38:43.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:38:44.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836494 2026-02-17T11:38:44.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:38:45.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:38:45.114 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:38:45.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836494 -lt 21474836494 2026-02-17T11:38:45.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:38:45.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-141733920773 2026-02-17T11:38:45.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:38:45.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:38:45.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-141733920773 2026-02-17T11:38:45.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:38:45.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=141733920773 2026-02-17T11:38:45.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 141733920773' 2026-02-17T11:38:45.275 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 141733920773 2026-02-17T11:38:45.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:38:45.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920773 -lt 141733920773 2026-02-17T11:38:45.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:38:45.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542155 2026-02-17T11:38:45.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:38:45.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:38:45.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542155 2026-02-17T11:38:45.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:38:45.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542155 2026-02-17T11:38:45.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542155' 2026-02-17T11:38:45.430 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542155 2026-02-17T11:38:45.430 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:38:45.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542155 -lt 60129542155 2026-02-17T11:38:45.588 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:38:45.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:38:45.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:38:45.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:38:45.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:38:45.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:38:45.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:38:45.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:38:45.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:38:45.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:38:45.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:38:45.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:38:45.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:38:45.931 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:38:45.931 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:38:46.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:38:46.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:38:46.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:38:46.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: auto_repair_erasure_coded: get_not_primary ecpool SOMETHING 2026-02-17T11:38:46.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=ecpool 2026-02-17T11:38:46.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:38:46.117 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary ecpool SOMETHING 2026-02-17T11:38:46.117 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T11:38:46.117 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:38:46.118 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:38:46.118 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:38:46.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=2 2026-02-17T11:38:46.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:38:46.274 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 2)) | .[0]' 2026-02-17T11:38:46.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:303: auto_repair_erasure_coded: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:38:46.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:38:46.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:38:46.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:38:46.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:38:46.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:38:46.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:38:46.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:38:46.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:38:46.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:38:46.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:38:46.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:38:46.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:38:46.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:38:46.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:38:46.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:38:47.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:38:47.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:38:47.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:38:47.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:38:47.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:38:47.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:38:47.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:38:47.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T11:38:48.052 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::scrub_1.3:head#, (61) No data available 2026-02-17T11:38:48.053 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:38:48.053 INFO:tasks.workunit.client.0.trial128.stdout:hinfo_key 2026-02-17T11:38:48.053 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:38:48.054 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#-3:00000000:::scrub_1.0:head#, (61) No data available 2026-02-17T11:38:48.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:38:48.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:38:48.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:38:48.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:38:48.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:38:48.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:38:48.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:38:48.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:38:48.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:38:48.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:38:48.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:38:48.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:38:48.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:38:48.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:38:48.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:38:48.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:38:48.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:38:48.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:38:48.589 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:38:48.589 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:38:48.589 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:38:48.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:38:48.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:38:48.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:38:48.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:38:48.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:38:48.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:38:48.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:38:48.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:38:48.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:38:48.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:38:48.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T11:38:48.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:38:48.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:38:48.593 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:38:48.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-auto-repair=true --osd-deep-scrub-interval=5 --osd-scrub-max-interval=5 --osd-scrub-min-interval=5 --osd-scrub-interval-randomize-ratio=0 2026-02-17T11:38:48.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:38:48.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:38:48.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:38:48.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:38:48.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:38:48.608 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:48.599+0000 7f1a80f0c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:48.609 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:48.599+0000 7f1a80f0c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:48.610 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:48.603+0000 7f1a80f0c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:48.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:38:48.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:38:48.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:38:48.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:38:48.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:38:48.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:48.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:38:48.741 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:38:48.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:48.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:38:48.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:38:49.864 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:49.855+0000 7f1a80f0c8c0 -1 Falling back to public interface 2026-02-17T11:38:49.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:49.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:49.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:38:49.898 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:38:49.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:49.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:38:50.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:38:51.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:51.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:51.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:38:51.054 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:38:51.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:51.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:38:51.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:38:52.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:52.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:52.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:38:52.210 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:38:52.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:52.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:38:52.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:38:52.880 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:52.871+0000 7f1a80f0c8c0 -1 osd.1 36 log_to_monitors true 2026-02-17T11:38:53.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:38:53.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:53.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:38:53.364 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:38:53.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:53.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:38:53.517 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 40 up_thru 40 down_at 37 last_clean_interval [33,36) [v2:127.0.0.1:6810/1259142432,v1:127.0.0.1:6811/1259142432] [v2:127.0.0.1:6812/1259142432,v1:127.0.0.1:6813/1259142432] exists,up f25cefe1-5783-431d-bc58-9f267f54f9f2 2026-02-17T11:38:53.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:38:53.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:38:53.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:38:53.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:38:53.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:38:53.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:38:53.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:38:53.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:38:53.519 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:38:53.519 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:38:53.519 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:38:53.520 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:38:53.520 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:38:53.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:38:53.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:38:53.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:38:53.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:38:53.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:38:53.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:38:53.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:38:53.780 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:38:53.780 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:38:53.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:38:53.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:38:53.781 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:38:53.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836497 2026-02-17T11:38:53.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836497 2026-02-17T11:38:53.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497' 2026-02-17T11:38:53.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:38:53.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:38:53.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691842 2026-02-17T11:38:53.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691842 2026-02-17T11:38:53.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-171798691842' 2026-02-17T11:38:53.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:38:53.925 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:38:53.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542158 2026-02-17T11:38:53.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542158 2026-02-17T11:38:53.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-171798691842 2-60129542158' 2026-02-17T11:38:53.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:38:53.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836497 2026-02-17T11:38:53.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:38:53.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:38:53.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836497 2026-02-17T11:38:53.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:38:54.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836497 2026-02-17T11:38:54.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2026-02-17T11:38:54.001 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836497 2026-02-17T11:38:54.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:38:54.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836497 2026-02-17T11:38:54.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:38:55.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:38:55.162 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:38:55.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836497 -lt 21474836497 2026-02-17T11:38:55.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:38:55.321 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-171798691842 2026-02-17T11:38:55.321 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:38:55.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:38:55.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-171798691842 2026-02-17T11:38:55.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:38:55.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691842 2026-02-17T11:38:55.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 171798691842' 2026-02-17T11:38:55.324 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 171798691842 2026-02-17T11:38:55.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:38:55.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691842 -lt 171798691842 2026-02-17T11:38:55.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:38:55.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542158 2026-02-17T11:38:55.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:38:55.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:38:55.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542158 2026-02-17T11:38:55.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:38:55.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542158 2026-02-17T11:38:55.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542158' 2026-02-17T11:38:55.486 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542158 2026-02-17T11:38:55.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:38:55.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542158 -lt 60129542158 2026-02-17T11:38:55.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:38:55.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:38:55.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:38:55.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:38:55.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:38:55.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:38:55.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:38:55.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:38:55.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:38:55.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:38:55.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:38:55.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:38:55.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:38:55.980 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:38:55.980 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:38:56.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:38:56.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:38:56.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:38:56.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:304: auto_repair_erasure_coded: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:38:56.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:305: auto_repair_erasure_coded: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:38:56.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T11:38:56.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:38:56.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:38:56.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:38:56.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:38:56.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:38:56.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:38:56.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:38:56.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:38:56.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:38:56.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:38:56.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:38:56.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:38:56.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:38:56.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:38:56.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:38:56.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:38:56.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:38:56.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:38:56.295 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:38:56.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:38:56.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:38:56.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:38:56.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:38:56.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:38:56.314 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:38:56.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:38:56.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:38:56.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:38:56.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T11:38:56.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T11:38:56.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T11:38:56.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T11:38:56.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:38:56.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:38:56.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:38:56.319 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:38:56.319 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:38:56.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:38:56.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:38:56.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:38:56.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:38:56.322 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:38:56.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:38:56.322 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:38:56.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:38:56.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:38:56.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:38:56.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:38:56.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:38:56.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:38:56.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:38:56.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:38:56.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:38:56.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:38:56.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:38:56.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:38:56.329 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:38:56.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:38:56.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:38:56.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:38:56.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T11:38:56.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:38:56.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:38:56.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:38:56.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T11:38:56.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:38:56.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T11:38:56.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:38:56.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T11:38:56.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_and_repair_jerasure_appends td/osd-scrub-repair 2026-02-17T11:38:56.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:815: TEST_corrupt_and_repair_jerasure_appends: corrupt_and_repair_jerasure td/osd-scrub-repair false 2026-02-17T11:38:56.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:798: corrupt_and_repair_jerasure: local dir=td/osd-scrub-repair 2026-02-17T11:38:56.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:799: corrupt_and_repair_jerasure: local allow_overwrites=false 2026-02-17T11:38:56.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:800: corrupt_and_repair_jerasure: local poolname=ecpool 2026-02-17T11:38:56.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:802: corrupt_and_repair_jerasure: run_mon td/osd-scrub-repair a 2026-02-17T11:38:56.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T11:38:56.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T11:38:56.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T11:38:56.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T11:38:56.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T11:38:56.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T11:38:56.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T11:38:56.355 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:38:56.355 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:38:56.355 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:38:56.355 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:38:56.355 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:38:56.355 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:38:56.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T11:38:56.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T11:38:56.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T11:38:56.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:38:56.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:38:56.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T11:38:56.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T11:38:56.385 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:38:56.385 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:38:56.385 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:38:56.385 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:38:56.385 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:38:56.385 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:38:56.386 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:38:56.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:38:56.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T11:38:56.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T11:38:56.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:38:56.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:38:56.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T11:38:56.443 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T11:38:56.443 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:38:56.443 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:38:56.443 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:38:56.443 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:38:56.443 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:38:56.443 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:38:56.444 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:38:56.444 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:38:56.444 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T11:38:56.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:803: corrupt_and_repair_jerasure: run_mgr td/osd-scrub-repair x 2026-02-17T11:38:56.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T11:38:56.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T11:38:56.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T11:38:56.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T11:38:56.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T11:38:56.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T11:38:56.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T11:38:56.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:38:56.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:38:56.598 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:38:56.598 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:38:56.598 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:38:56.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:38:56.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:38:56.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:38:56.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: seq 0 3 2026-02-17T11:38:56.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-17T11:38:56.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 0 2026-02-17T11:38:56.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:38:56.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:38:56.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T11:38:56.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:38:56.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:38:56.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:38:56.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:38:56.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:38:56.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:38:56.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:38:56.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:38:56.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:38:56.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:38:56.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:38:56.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:38:56.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:38:56.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:38:56.621 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:38:56.621 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:38:56.621 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:38:56.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:38:56.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:38:56.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:38:56.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:38:56.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:38:56.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:38:56.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:38:56.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:38:56.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:38:56.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:38:56.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:38:56.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:38:56.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:38:56.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:38:56.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:38:56.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=0ba46996-3a87-4b8f-b85b-257bb87f5357 2026-02-17T11:38:56.623 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 0ba46996-3a87-4b8f-b85b-257bb87f5357 2026-02-17T11:38:56.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 0ba46996-3a87-4b8f-b85b-257bb87f5357' 2026-02-17T11:38:56.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:38:56.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBQU5Rpg3BNJRAAS3RaRawEOMQP6iSRhFc06Q== 2026-02-17T11:38:56.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBQU5Rpg3BNJRAAS3RaRawEOMQP6iSRhFc06Q=="}' 2026-02-17T11:38:56.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 0ba46996-3a87-4b8f-b85b-257bb87f5357 -i td/osd-scrub-repair/0/new.json 2026-02-17T11:38:56.720 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:38:56.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T11:38:56.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBQU5Rpg3BNJRAAS3RaRawEOMQP6iSRhFc06Q== --osd-uuid 0ba46996-3a87-4b8f-b85b-257bb87f5357 2026-02-17T11:38:56.744 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:56.735+0000 7fec7bb2a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:56.746 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:56.735+0000 7fec7bb2a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:56.747 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:56.739+0000 7fec7bb2a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:56.748 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:56.739+0000 7fec7bb2a8c0 -1 bdev(0x56262aa6fc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:38:56.748 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:56.739+0000 7fec7bb2a8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T11:38:59.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T11:38:59.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:38:59.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T11:38:59.364 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:38:59.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:38:59.564 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:38:59.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T11:38:59.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:38:59.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:38:59.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:38:59.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:38:59.587 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:59.579+0000 7fb9b06148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:59.589 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:59.579+0000 7fb9b06148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:59.590 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:38:59.583+0000 7fb9b06148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:38:59.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T11:38:59.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:38:59.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:38:59.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:38:59.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:38:59.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:38:59.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:38:59.716 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:38:59.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:38:59.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:38:59.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:00.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:00.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:00.873 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:39:00.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:39:00.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:00.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:39:01.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:01.300 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:01.291+0000 7fb9b06148c0 -1 Falling back to public interface 2026-02-17T11:39:02.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:02.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:02.029 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:39:02.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:39:02.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:02.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:39:02.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:03.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:03.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:03.183 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:39:03.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:39:03.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:03.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:39:03.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:04.306 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:04.295+0000 7fb9b06148c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:39:04.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:04.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:04.340 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:39:04.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:39:04.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:04.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:39:04.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:05.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:05.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:05.495 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:39:05.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:39:05.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:05.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:39:05.650 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/585750724,v1:127.0.0.1:6803/585750724] [v2:127.0.0.1:6804/585750724,v1:127.0.0.1:6805/585750724] exists,up 0ba46996-3a87-4b8f-b85b-257bb87f5357 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 1 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:39:05.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:39:05.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:39:05.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:39:05.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:39:05.652 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:39:05.652 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:39:05.652 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:39:05.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:39:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:39:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:39:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:39:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:39:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:39:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:39:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:39:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:39:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:39:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:39:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:39:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:39:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:39:05.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:39:05.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2f7d8768-732b-4677-9611-0afb8ab0865e 2026-02-17T11:39:05.656 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 2f7d8768-732b-4677-9611-0afb8ab0865e 2026-02-17T11:39:05.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 2f7d8768-732b-4677-9611-0afb8ab0865e' 2026-02-17T11:39:05.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:39:05.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBZU5RpC8SHJxAA/H3E+E3g9kfVcteD67rg3Q== 2026-02-17T11:39:05.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBZU5RpC8SHJxAA/H3E+E3g9kfVcteD67rg3Q=="}' 2026-02-17T11:39:05.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2f7d8768-732b-4677-9611-0afb8ab0865e -i td/osd-scrub-repair/1/new.json 2026-02-17T11:39:05.827 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:39:05.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T11:39:05.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBZU5RpC8SHJxAA/H3E+E3g9kfVcteD67rg3Q== --osd-uuid 2f7d8768-732b-4677-9611-0afb8ab0865e 2026-02-17T11:39:05.852 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:05.843+0000 7fad78cc68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:05.854 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:05.843+0000 7fad78cc68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:05.855 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:05.847+0000 7fad78cc68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:05.856 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:05.847+0000 7fad78cc68c0 -1 bdev(0x5625acb45c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:39:05.856 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:05.847+0000 7fad78cc68c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T11:39:08.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T11:39:08.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:39:08.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T11:39:08.153 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:39:08.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:39:08.348 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:39:08.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T11:39:08.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:39:08.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:39:08.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:39:08.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:39:08.371 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:08.363+0000 7fd6c63e18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:08.372 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:08.363+0000 7fd6c63e18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:08.373 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:08.363+0000 7fd6c63e18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:08.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T11:39:08.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:39:08.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:39:08.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:39:08.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:39:08.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:08.501 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:39:08.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:39:08.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:08.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:39:08.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:09.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:09.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:09.657 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:39:09.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:39:09.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:09.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:39:09.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:09.856 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:09.847+0000 7fd6c63e18c0 -1 Falling back to public interface 2026-02-17T11:39:10.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:10.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:10.815 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:39:10.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:39:10.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:10.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:39:10.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:11.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:11.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:11.970 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:39:11.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:39:11.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:11.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:39:12.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:12.882 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:12.871+0000 7fd6c63e18c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:39:13.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:13.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:13.126 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:39:13.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:39:13.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:13.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:39:13.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:14.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:14.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:14.284 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:39:14.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:39:14.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:14.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:39:14.435 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4123282179,v1:127.0.0.1:6811/4123282179] [v2:127.0.0.1:6812/4123282179,v1:127.0.0.1:6813/4123282179] exists,up 2f7d8768-732b-4677-9611-0afb8ab0865e 2026-02-17T11:39:14.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:39:14.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:39:14.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:39:14.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-17T11:39:14.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 2 2026-02-17T11:39:14.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:39:14.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:39:14.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T11:39:14.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:39:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:39:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:39:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:39:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:39:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:39:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:39:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:39:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:39:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:39:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:39:14.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:39:14.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:39:14.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:39:14.441 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:39:14.441 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:39:14.441 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:39:14.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:39:14.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:39:14.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:39:14.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:39:14.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:39:14.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:39:14.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:39:14.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:39:14.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:39:14.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:39:14.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:39:14.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:39:14.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:39:14.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:39:14.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:39:14.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=56d7ecf7-cec9-44c9-807a-2f5f5432a860 2026-02-17T11:39:14.443 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 56d7ecf7-cec9-44c9-807a-2f5f5432a860 2026-02-17T11:39:14.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 56d7ecf7-cec9-44c9-807a-2f5f5432a860' 2026-02-17T11:39:14.444 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:39:14.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBiU5RpR/yLGhAAxlOxHHyN+TjPhRNn1oPn7w== 2026-02-17T11:39:14.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBiU5RpR/yLGhAAxlOxHHyN+TjPhRNn1oPn7w=="}' 2026-02-17T11:39:14.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 56d7ecf7-cec9-44c9-807a-2f5f5432a860 -i td/osd-scrub-repair/2/new.json 2026-02-17T11:39:14.596 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:39:14.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T11:39:14.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBiU5RpR/yLGhAAxlOxHHyN+TjPhRNn1oPn7w== --osd-uuid 56d7ecf7-cec9-44c9-807a-2f5f5432a860 2026-02-17T11:39:14.622 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:14.611+0000 7f43cc7d58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:14.624 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:14.615+0000 7f43cc7d58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:14.625 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:14.615+0000 7f43cc7d58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:14.625 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:14.615+0000 7f43cc7d58c0 -1 bdev(0x55725acbfc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T11:39:14.626 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:14.615+0000 7f43cc7d58c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T11:39:16.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T11:39:16.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:39:16.896 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T11:39:16.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T11:39:16.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:39:17.095 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T11:39:17.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T11:39:17.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:39:17.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:39:17.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:39:17.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:39:17.111 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:17.103+0000 7fbd896ae8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:17.113 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:17.103+0000 7fbd896ae8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:17.114 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:17.103+0000 7fbd896ae8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:17.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T11:39:17.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:39:17.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:39:17.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:39:17.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:39:17.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:17.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:39:17.247 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:39:17.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:17.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:39:17.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:18.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:18.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:18.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:39:18.404 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:39:18.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:18.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:39:18.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:18.612 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:18.603+0000 7fbd896ae8c0 -1 Falling back to public interface 2026-02-17T11:39:19.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:19.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:19.559 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:39:19.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:39:19.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:19.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:39:19.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:20.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:20.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:20.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:39:20.709 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:39:20.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:20.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:39:20.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:21.642 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:21.630+0000 7fbd896ae8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T11:39:21.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:21.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:21.858 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:39:21.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:39:21.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:21.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:39:22.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:23.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:23.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:23.018 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:39:23.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:39:23.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:23.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:39:23.167 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3819270548,v1:127.0.0.1:6819/3819270548] [v2:127.0.0.1:6820/3819270548,v1:127.0.0.1:6821/3819270548] exists,up 56d7ecf7-cec9-44c9-807a-2f5f5432a860 2026-02-17T11:39:23.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:39:23.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:39:23.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:39:23.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-17T11:39:23.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 3 2026-02-17T11:39:23.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:39:23.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:39:23.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T11:39:23.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:39:23.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:39:23.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:39:23.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:39:23.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:39:23.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:39:23.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:39:23.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:39:23.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:39:23.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:39:23.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:39:23.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:39:23.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:39:23.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:39:23.173 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:39:23.173 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:39:23.173 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:39:23.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:39:23.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:39:23.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:39:23.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:39:23.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:39:23.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:39:23.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:39:23.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:39:23.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:39:23.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:39:23.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:39:23.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:39:23.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:39:23.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:39:23.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:39:23.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2ce21662-caa1-40c6-be78-a6b6218bdef1 2026-02-17T11:39:23.175 INFO:tasks.workunit.client.0.trial128.stdout:add osd3 2ce21662-caa1-40c6-be78-a6b6218bdef1 2026-02-17T11:39:23.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 2ce21662-caa1-40c6-be78-a6b6218bdef1' 2026-02-17T11:39:23.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:39:23.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBrU5RpU8SmChAAwU4SVvHPco7rDhUa0mdd1A== 2026-02-17T11:39:23.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBrU5RpU8SmChAAwU4SVvHPco7rDhUa0mdd1A=="}' 2026-02-17T11:39:23.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2ce21662-caa1-40c6-be78-a6b6218bdef1 -i td/osd-scrub-repair/3/new.json 2026-02-17T11:39:23.337 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:39:23.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-17T11:39:23.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBrU5RpU8SmChAAwU4SVvHPco7rDhUa0mdd1A== --osd-uuid 2ce21662-caa1-40c6-be78-a6b6218bdef1 2026-02-17T11:39:23.362 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:23.350+0000 7efcec7e48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:23.363 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:23.354+0000 7efcec7e48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:23.364 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:23.354+0000 7efcec7e48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:23.365 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:23.354+0000 7efcec7e48c0 -1 bdev(0x55a76a61bc00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-17T11:39:23.365 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:23.354+0000 7efcec7e48c0 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-17T11:39:25.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-17T11:39:25.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:39:25.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T11:39:25.644 INFO:tasks.workunit.client.0.trial128.stdout:adding osd3 key to auth repository 2026-02-17T11:39:25.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:39:25.834 INFO:tasks.workunit.client.0.trial128.stdout:start osd.3 2026-02-17T11:39:25.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T11:39:25.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:39:25.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:39:25.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:39:25.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:39:25.858 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:25.846+0000 7f783bbce8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:25.860 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:25.850+0000 7f783bbce8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:25.861 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:25.850+0000 7f783bbce8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:25.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T11:39:25.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:39:25.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:39:25.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:39:25.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:39:25.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:25.990 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:39:25.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:39:25.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:25.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:39:26.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:27.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:27.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:27.141 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:39:27.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:39:27.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:27.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:39:27.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:27.344 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:27.334+0000 7f783bbce8c0 -1 Falling back to public interface 2026-02-17T11:39:28.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:28.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:28.299 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:39:28.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:39:28.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:28.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:39:28.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:29.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:29.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:29.450 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:39:29.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:39:29.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:29.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:39:29.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:30.410 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:30.398+0000 7f783bbce8c0 -1 osd.3 0 log_to_monitors true 2026-02-17T11:39:30.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:30.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:30.599 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:39:30.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:39:30.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:30.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:39:30.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:31.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:31.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:31.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:39:31.754 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:39:31.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:31.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:39:31.901 INFO:tasks.workunit.client.0.trial128.stdout:osd.3 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1599414675,v1:127.0.0.1:6827/1599414675] [v2:127.0.0.1:6828/1599414675,v1:127.0.0.1:6829/1599414675] exists,up 2ce21662-caa1-40c6-be78-a6b6218bdef1 2026-02-17T11:39:31.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:39:31.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:39:31.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:39:31.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:807: corrupt_and_repair_jerasure: create_rbd_pool 2026-02-17T11:39:31.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T11:39:32.042 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' does not exist 2026-02-17T11:39:32.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T11:39:32.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T11:39:32.250 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T11:39:32.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:39:33.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T11:39:33.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: corrupt_and_repair_jerasure: wait_for_clean 2026-02-17T11:39:33.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:39:33.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:39:33.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:39:33.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:39:33.544 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:39:33.544 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:39:33.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:39:33.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:39:33.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:39:33.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:39:33.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:39:33.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:39:33.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:39:33.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:39:33.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:39:33.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:39:33.809 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:39:33.809 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:39:33.809 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T11:39:33.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:39:33.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:39:33.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:39:33.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T11:39:33.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T11:39:33.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T11:39:33.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:39:33.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:39:33.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672965 2026-02-17T11:39:33.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672965 2026-02-17T11:39:33.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672965' 2026-02-17T11:39:33.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:39:33.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:39:34.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542148 2026-02-17T11:39:34.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542148 2026-02-17T11:39:34.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672965 2-60129542148' 2026-02-17T11:39:34.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:39:34.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:39:34.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=81604378626 2026-02-17T11:39:34.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 81604378626 2026-02-17T11:39:34.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-42949672965 2-60129542148 3-81604378626' 2026-02-17T11:39:34.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:39:34.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T11:39:34.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:39:34.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:39:34.099 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T11:39:34.099 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:39:34.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T11:39:34.101 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836487 2026-02-17T11:39:34.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T11:39:34.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:39:34.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-17T11:39:34.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:39:35.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:39:35.260 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:39:35.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T11:39:35.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:39:35.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672965 2026-02-17T11:39:35.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:39:35.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:39:35.421 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672965 2026-02-17T11:39:35.421 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:39:35.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672965 2026-02-17T11:39:35.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672965' 2026-02-17T11:39:35.423 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672965 2026-02-17T11:39:35.423 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:39:35.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672966 -lt 42949672965 2026-02-17T11:39:35.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:39:35.583 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542148 2026-02-17T11:39:35.583 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:39:35.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:39:35.585 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542148 2026-02-17T11:39:35.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:39:35.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542148 2026-02-17T11:39:35.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-17T11:39:35.587 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542148 2026-02-17T11:39:35.587 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:39:35.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-17T11:39:35.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:39:35.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-81604378626 2026-02-17T11:39:35.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:39:35.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:39:35.737 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-81604378626 2026-02-17T11:39:35.737 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:39:35.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=81604378626 2026-02-17T11:39:35.738 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.3 seq 81604378626 2026-02-17T11:39:35.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 81604378626' 2026-02-17T11:39:35.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:39:35.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 81604378626 -lt 81604378626 2026-02-17T11:39:35.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:39:35.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:39:35.906 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:39:36.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T11:39:36.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:39:36.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:39:36.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:39:36.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:39:36.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:39:36.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:39:36.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:39:36.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T11:39:36.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:39:36.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:39:36.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:39:36.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T11:39:36.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:39:36.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:39:36.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:810: corrupt_and_repair_jerasure: create_ec_pool ecpool false k=2 m=2 2026-02-17T11:39:36.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T11:39:36.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T11:39:36.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=false 2026-02-17T11:39:36.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T11:39:36.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=2 2026-02-17T11:39:36.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T11:39:36.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T11:39:36.892 INFO:tasks.workunit.client.0.trial128.stderr:pool 'ecpool' created 2026-02-17T11:39:36.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:39:37.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' false = true ']' 2026-02-17T11:39:37.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T11:39:37.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:39:37.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:39:37.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:39:37.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:39:37.906 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:39:37.906 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:39:37.907 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:39:37.907 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:39:37.907 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:39:38.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:39:38.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:39:38.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:39:38.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:39:38.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:39:38.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:39:38.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:39:38.166 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:39:38.166 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:39:38.166 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T11:39:38.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:39:38.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:39:38.166 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:39:38.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-17T11:39:38.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-17T11:39:38.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-17T11:39:38.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:39:38.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:39:38.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672967 2026-02-17T11:39:38.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672967 2026-02-17T11:39:38.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-42949672967' 2026-02-17T11:39:38.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:39:38.314 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:39:38.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542150 2026-02-17T11:39:38.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542150 2026-02-17T11:39:38.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-42949672967 2-60129542150' 2026-02-17T11:39:38.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:39:38.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:39:38.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=81604378628 2026-02-17T11:39:38.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 81604378628 2026-02-17T11:39:38.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-42949672967 2-60129542150 3-81604378628' 2026-02-17T11:39:38.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:39:38.457 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-17T11:39:38.457 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:39:38.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:39:38.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-17T11:39:38.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:39:38.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-17T11:39:38.461 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836489 2026-02-17T11:39:38.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-17T11:39:38.461 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:39:38.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-17T11:39:38.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:39:39.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:39:39.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:39:39.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-17T11:39:39.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:39:39.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672967 2026-02-17T11:39:39.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:39:39.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:39:39.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672967 2026-02-17T11:39:39.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:39:39.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672967 2026-02-17T11:39:39.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672967' 2026-02-17T11:39:39.774 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672967 2026-02-17T11:39:39.774 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:39:39.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672967 -lt 42949672967 2026-02-17T11:39:39.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:39:39.925 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542150 2026-02-17T11:39:39.925 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:39:39.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:39:39.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542150 2026-02-17T11:39:39.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:39:39.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542150 2026-02-17T11:39:39.929 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542150 2026-02-17T11:39:39.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-17T11:39:39.929 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:39:40.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-17T11:39:40.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:39:40.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-81604378628 2026-02-17T11:39:40.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:39:40.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:39:40.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-81604378628 2026-02-17T11:39:40.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:39:40.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=81604378628 2026-02-17T11:39:40.097 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.3 seq 81604378628 2026-02-17T11:39:40.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 81604378628' 2026-02-17T11:39:40.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:39:40.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 81604378628 -lt 81604378628 2026-02-17T11:39:40.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:39:40.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:39:40.257 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:39:40.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:39:40.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:39:40.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:39:40.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:39:40.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:39:40.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:39:40.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:39:40.437 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:39:40.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:39:40.585 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:39:40.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:39:40.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:39:40.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:39:40.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:39:40.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:39:40.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T11:39:40.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:811: corrupt_and_repair_jerasure: corrupt_and_repair_erasure_coded td/osd-scrub-repair ecpool 2026-02-17T11:39:40.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:248: corrupt_and_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T11:39:40.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:249: corrupt_and_repair_erasure_coded: local poolname=ecpool 2026-02-17T11:39:40.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:251: corrupt_and_repair_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-17T11:39:40.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:39:40.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:39:40.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T11:39:40.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:39:40.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:39:40.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:39:40.961 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:39:40.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:39:41.165 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:39:41.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:39:41.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:39:41.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T11:39:41.201 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_erasure_coded: get_primary ecpool SOMETHING 2026-02-17T11:39:41.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T11:39:41.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:39:41.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:39:41.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:39:41.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_erasure_coded: local primary=3 2026-02-17T11:39:41.353 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: get_osds ecpool SOMETHING 2026-02-17T11:39:41.353 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T11:39:41.353 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: sed -e s/3// 2026-02-17T11:39:41.353 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T11:39:41.353 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:39:41.354 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T11:39:41.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T11:39:41.510 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:39:41.510 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:39:41.510 INFO:tasks.workunit.client.0.trial128.stderr:0' 2026-02-17T11:39:41.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 0 2026-02-17T11:39:41.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: osds=('1' '2' '0') 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: local -a osds 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:255: corrupt_and_repair_erasure_coded: local not_primary_first=1 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:256: corrupt_and_repair_erasure_coded: local not_primary_second=2 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:259: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 3 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=ecpool 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=3 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:39:41.512 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:39:41.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:39:41.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:39:41.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:39:42.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:39:42.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:39:42.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:39:42.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:39:42.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T11:39:42.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:39:42.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:39:42.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T11:39:43.457 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#2:eb822e21:::SOMETHING:head# 2026-02-17T11:39:43.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T11:39:43.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:39:43.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:39:43.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T11:39:43.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:39:43.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:39:43.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:39:43.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:39:43.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:39:43.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:39:43.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:39:43.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:39:43.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:39:43.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:39:43.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:39:43.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:39:43.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:39:43.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:39:43.996 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:39:43.996 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:39:43.996 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:39:43.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:39:43.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:39:43.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:39:43.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:39:43.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:39:43.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:39:43.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:39:43.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:39:43.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:39:43.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:39:43.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:39:43.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:39:43.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T11:39:43.998 INFO:tasks.workunit.client.0.trial128.stderr:start osd.3 2026-02-17T11:39:43.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:39:43.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T11:39:43.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T11:39:43.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:39:43.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:39:43.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:39:44.013 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:44.002+0000 7fec1ed568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:44.014 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:44.002+0000 7fec1ed568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:44.015 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:44.006+0000 7fec1ed568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:39:44.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T11:39:44.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:39:44.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:39:44.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:39:44.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:39:44.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:44.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:39:44.148 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:39:44.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:44.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:39:44.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:45.245 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:45.234+0000 7fec1ed568c0 -1 Falling back to public interface 2026-02-17T11:39:45.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:45.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:45.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:39:45.302 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:39:45.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:45.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:39:45.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:46.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:46.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:46.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:39:46.463 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:39:46.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:46.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:39:46.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:47.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:47.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:47.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:39:47.622 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:39:47.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:47.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:39:47.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:39:48.219 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:39:48.210+0000 7fec1ed568c0 -1 osd.3 35 log_to_monitors true 2026-02-17T11:39:48.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:39:48.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:39:48.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:39:48.778 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:39:48.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:39:48.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:39:48.931 INFO:tasks.workunit.client.0.trial128.stderr:osd.3 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [19,35) [v2:127.0.0.1:6826/250275263,v1:127.0.0.1:6827/250275263] [v2:127.0.0.1:6828/250275263,v1:127.0.0.1:6829/250275263] exists,up 2ce21662-caa1-40c6-be78-a6b6218bdef1 2026-02-17T11:39:48.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:39:48.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:39:48.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:39:48.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:39:48.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:39:48.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:39:48.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:39:48.932 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:39:48.932 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:39:48.933 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:39:48.933 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:39:48.933 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:39:48.934 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:39:49.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:39:49.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:39:49.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:39:49.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:39:49.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:39:49.042 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:39:49.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:39:49.194 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:39:49.194 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:39:49.194 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T11:39:49.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:39:49.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:39:49.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:39:49.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836492 2026-02-17T11:39:49.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836492 2026-02-17T11:39:49.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492' 2026-02-17T11:39:49.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:39:49.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:39:49.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672971 2026-02-17T11:39:49.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672971 2026-02-17T11:39:49.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-42949672971' 2026-02-17T11:39:49.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:39:49.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:39:49.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542153 2026-02-17T11:39:49.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542153 2026-02-17T11:39:49.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542153' 2026-02-17T11:39:49.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:39:49.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:39:49.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724546 2026-02-17T11:39:49.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724546 2026-02-17T11:39:49.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-42949672971 2-60129542153 3-167503724546' 2026-02-17T11:39:49.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:39:49.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836492 2026-02-17T11:39:49.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:39:49.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:39:49.483 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836492 2026-02-17T11:39:49.483 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:39:49.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836492 2026-02-17T11:39:49.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-17T11:39:49.484 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836492 2026-02-17T11:39:49.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:39:49.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836492 2026-02-17T11:39:49.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:39:50.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:39:50.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:39:50.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836492 2026-02-17T11:39:50.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:39:51.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:39:51.801 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:39:51.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836492 2026-02-17T11:39:51.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:39:51.956 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672971 2026-02-17T11:39:51.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:39:51.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:39:51.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672971 2026-02-17T11:39:51.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:39:51.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672971 2026-02-17T11:39:51.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672971' 2026-02-17T11:39:51.960 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 42949672971 2026-02-17T11:39:51.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:39:52.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672971 -lt 42949672971 2026-02-17T11:39:52.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:39:52.111 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542153 2026-02-17T11:39:52.111 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:39:52.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:39:52.113 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542153 2026-02-17T11:39:52.113 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:39:52.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542153 2026-02-17T11:39:52.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-17T11:39:52.115 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542153 2026-02-17T11:39:52.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:39:52.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-17T11:39:52.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:39:52.274 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-167503724546 2026-02-17T11:39:52.274 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:39:52.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:39:52.276 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-167503724546 2026-02-17T11:39:52.276 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:39:52.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724546 2026-02-17T11:39:52.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 167503724546' 2026-02-17T11:39:52.277 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.3 seq 167503724546 2026-02-17T11:39:52.278 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:39:52.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724546 -lt 167503724546 2026-02-17T11:39:52.434 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:39:52.434 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:39:52.434 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:39:52.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:39:52.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:39:52.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:39:52.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:39:52.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:39:52.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:39:52.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:39:52.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:39:52.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:39:52.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:39:52.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:39:52.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:39:52.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:39:52.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:39:52.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:39:52.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-17T11:39:52.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:39:52.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:39:52.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:39:52.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:39:53.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=2.0 2026-02-17T11:39:53.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 2.0 2026-02-17T11:39:53.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T11:39:53.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T11:39:53.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:39:53.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:39:53.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:39:53.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:39:53.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:39:36.885294+0000 2026-02-17T11:39:53.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T11:39:53.401 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T11:39:53.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T11:39:36.885294+0000 2026-02-17T11:39:53.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:39:53.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:39:36.885294+0000 2026-02-17T11:39:53.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:39:53.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:39:53.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:39:53.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:39:53.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:39:53.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:39:53.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:39:53.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:39:53.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:39:36.885294+0000 '>' 2026-02-17T11:39:36.885294+0000 2026-02-17T11:39:53.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:39:54.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:39:54.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:39:54.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:39:54.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:39:54.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:39:54.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:39:54.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:39:54.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:39:36.885294+0000 '>' 2026-02-17T11:39:36.885294+0000 2026-02-17T11:39:54.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:39:55.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:39:55.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:39:55.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:39:55.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:39:55.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:39:55.726 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:39:55.726 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:39:55.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:39:36.885294+0000 '>' 2026-02-17T11:39:36.885294+0000 2026-02-17T11:39:55.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:39:56.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:39:56.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:39:56.876 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:39:56.876 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:39:56.876 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:39:56.876 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:39:56.876 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:39:57.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:39:36.885294+0000 '>' 2026-02-17T11:39:36.885294+0000 2026-02-17T11:39:57.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:39:58.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:39:58.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:39:58.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:39:58.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:39:58.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:39:58.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:39:58.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:39:58.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:39:36.885294+0000 '>' 2026-02-17T11:39:36.885294+0000 2026-02-17T11:39:58.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:39:59.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:39:59.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:39:59.188 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:39:59.188 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:39:59.188 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:39:59.188 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:39:59.188 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:39:59.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:39:54.176074+0000 '>' 2026-02-17T11:39:36.885294+0000 2026-02-17T11:39:59.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:39:59.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:39:59.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:39:59.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:39:59.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T11:39:59.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:39:59.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:39:59.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:39:59.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:39:59.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T11:39:59.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:39:59.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T11:39:59.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:39:59.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:39:59.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:39:59.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:39:59.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:40:00.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:40:00.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:40:00.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:40:00.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:40:00.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T11:40:00.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:40:00.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:40:00.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-17T11:40:00.962 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-17T11:40:00.962 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:40:00.962 INFO:tasks.workunit.client.0.trial128.stdout:hinfo_key 2026-02-17T11:40:00.962 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:40:01.495 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:40:01.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:40:01.496 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:40:01.496 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:40:01.496 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:40:01.497 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:40:01.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:40:01.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:40:01.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:40:01.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:40:01.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:40:01.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:40:01.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:40:01.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:40:01.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:40:01.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:40:01.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:40:01.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T11:40:01.498 INFO:tasks.workunit.client.0.trial128.stderr:start osd.3 2026-02-17T11:40:01.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:40:01.499 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T11:40:01.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T11:40:01.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:40:01.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:40:01.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:40:01.517 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:40:01.506+0000 7fe9cabca8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:01.518 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:40:01.510+0000 7fe9cabca8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:01.520 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:40:01.510+0000 7fe9cabca8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:01.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T11:40:01.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:40:01.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:40:01.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:40:01.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:40:01.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:01.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:40:01.653 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:40:01.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:01.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:40:01.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:02.760 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:40:02.750+0000 7fe9cabca8c0 -1 Falling back to public interface 2026-02-17T11:40:02.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:02.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:02.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:40:02.808 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:40:02.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:02.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:40:02.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:03.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:03.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:03.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:40:03.962 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:40:03.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:03.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:40:04.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:05.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:05.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:05.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:40:05.115 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:40:05.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:05.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:40:05.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:05.749 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:40:05.738+0000 7fe9cabca8c0 -1 osd.3 40 log_to_monitors true 2026-02-17T11:40:06.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:06.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:06.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:40:06.273 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:40:06.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:06.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:40:06.425 INFO:tasks.workunit.client.0.trial128.stderr:osd.3 up in weight 1 up_from 44 up_thru 44 down_at 41 last_clean_interval [39,40) [v2:127.0.0.1:6826/3264985406,v1:127.0.0.1:6827/3264985406] [v2:127.0.0.1:6828/3264985406,v1:127.0.0.1:6829/3264985406] exists,up 2ce21662-caa1-40c6-be78-a6b6218bdef1 2026-02-17T11:40:06.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:40:06.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:40:06.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:40:06.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:40:06.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:40:06.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:40:06.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:40:06.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:40:06.427 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:40:06.427 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:40:06.427 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:40:06.427 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:40:06.427 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:40:06.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:40:06.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:40:06.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:40:06.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:40:06.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:40:06.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:40:06.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:40:06.694 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:40:06.694 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:40:06.694 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T11:40:06.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:40:06.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:06.694 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:40:06.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836497 2026-02-17T11:40:06.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836497 2026-02-17T11:40:06.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497' 2026-02-17T11:40:06.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:06.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:40:06.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672975 2026-02-17T11:40:06.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672975 2026-02-17T11:40:06.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-42949672975' 2026-02-17T11:40:06.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:06.837 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:40:06.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542157 2026-02-17T11:40:06.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542157 2026-02-17T11:40:06.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-42949672975 2-60129542157' 2026-02-17T11:40:06.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:06.908 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:40:06.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561026 2026-02-17T11:40:06.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561026 2026-02-17T11:40:06.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-42949672975 2-60129542157 3-188978561026' 2026-02-17T11:40:06.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:06.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836497 2026-02-17T11:40:06.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:06.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:40:06.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836497 2026-02-17T11:40:06.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:06.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836497 2026-02-17T11:40:06.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2026-02-17T11:40:06.985 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836497 2026-02-17T11:40:06.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:40:07.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836497 -lt 21474836497 2026-02-17T11:40:07.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:07.146 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672975 2026-02-17T11:40:07.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:07.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:40:07.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:07.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672975 2026-02-17T11:40:07.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672975 2026-02-17T11:40:07.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672975' 2026-02-17T11:40:07.150 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 42949672975 2026-02-17T11:40:07.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:40:07.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672975 -lt 42949672975 2026-02-17T11:40:07.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:07.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542157 2026-02-17T11:40:07.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:07.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:40:07.312 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542157 2026-02-17T11:40:07.312 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:07.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542157 2026-02-17T11:40:07.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542157' 2026-02-17T11:40:07.314 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542157 2026-02-17T11:40:07.314 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:40:07.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542156 -lt 60129542157 2026-02-17T11:40:07.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:40:08.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:40:08.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:40:08.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542156 -lt 60129542157 2026-02-17T11:40:08.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:40:09.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:40:09.635 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:40:09.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542158 -lt 60129542157 2026-02-17T11:40:09.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:09.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-188978561026 2026-02-17T11:40:09.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:09.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:40:09.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-188978561026 2026-02-17T11:40:09.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:09.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561026 2026-02-17T11:40:09.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 188978561026' 2026-02-17T11:40:09.793 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.3 seq 188978561026 2026-02-17T11:40:09.793 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:40:09.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561026 -lt 188978561026 2026-02-17T11:40:09.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:40:09.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:40:09.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:40:10.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:40:10.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:40:10.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:40:10.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:40:10.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:40:10.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:40:10.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:40:10.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:40:10.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:40:10.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:40:10.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:40:10.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:40:10.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:40:10.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:40:10.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:40:10.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:40:10.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:40:10.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:261: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 1 2026-02-17T11:40:10.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T11:40:10.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=ecpool 2026-02-17T11:40:10.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=1 2026-02-17T11:40:10.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:40:10.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:40:10.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:40:10.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:40:10.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:40:10.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:40:10.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:40:10.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:40:10.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:40:10.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:40:10.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:40:10.495 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:40:10.495 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:40:10.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:40:10.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:40:10.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:40:11.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:40:11.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:40:11.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:40:11.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:40:11.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:40:11.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:40:11.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:40:11.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T11:40:12.457 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T11:40:12.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:40:12.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:40:12.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:40:12.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:40:12.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:40:12.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:40:12.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:40:12.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:40:12.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:40:12.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:40:12.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:40:12.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:40:12.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:40:12.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:40:12.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:40:12.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:40:12.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:40:12.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:40:12.996 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:40:12.996 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:40:12.996 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:40:12.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:40:12.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:40:12.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:40:12.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:40:12.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:40:12.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:40:12.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:40:12.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:40:12.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:40:12.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:40:12.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:40:12.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:40:12.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:40:12.999 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:40:12.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:40:12.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:40:13.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:40:13.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:40:13.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:40:13.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:40:13.019 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:40:13.009+0000 7f428a8e18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:13.020 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:40:13.009+0000 7f428a8e18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:13.021 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:40:13.009+0000 7f428a8e18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:13.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:40:13.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:40:13.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:40:13.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:40:13.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:40:13.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:13.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:40:13.151 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:40:13.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:13.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:40:13.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:14.273 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:40:14.265+0000 7f428a8e18c0 -1 Falling back to public interface 2026-02-17T11:40:14.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:14.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:14.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:40:14.307 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:40:14.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:14.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:40:14.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:15.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:15.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:15.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:40:15.461 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:40:15.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:15.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:40:15.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:16.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:16.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:16.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:40:16.613 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:40:16.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:16.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:40:16.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:17.260 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:40:17.253+0000 7f428a8e18c0 -1 osd.1 45 log_to_monitors true 2026-02-17T11:40:17.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:17.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:17.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:40:17.771 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:40:17.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:17.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:40:17.916 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 49 up_thru 49 down_at 46 last_clean_interval [10,45) [v2:127.0.0.1:6810/2681687567,v1:127.0.0.1:6811/2681687567] [v2:127.0.0.1:6812/2681687567,v1:127.0.0.1:6813/2681687567] exists,up 2f7d8768-732b-4677-9611-0afb8ab0865e 2026-02-17T11:40:17.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:40:17.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:40:17.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:40:17.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:40:17.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:40:17.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:40:17.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:40:17.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:40:17.917 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:40:17.917 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:40:17.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:40:17.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:40:17.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:40:18.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:40:18.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:40:18.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:40:18.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:40:18.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:40:18.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:40:18.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:40:18.186 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:40:18.186 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:40:18.186 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T11:40:18.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:40:18.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:18.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:40:18.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836500 2026-02-17T11:40:18.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836500 2026-02-17T11:40:18.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500' 2026-02-17T11:40:18.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:18.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:40:18.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=210453397506 2026-02-17T11:40:18.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 210453397506 2026-02-17T11:40:18.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-210453397506' 2026-02-17T11:40:18.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:18.329 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:40:18.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542161 2026-02-17T11:40:18.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542161 2026-02-17T11:40:18.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-210453397506 2-60129542161' 2026-02-17T11:40:18.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:18.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:40:18.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561029 2026-02-17T11:40:18.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561029 2026-02-17T11:40:18.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-210453397506 2-60129542161 3-188978561029' 2026-02-17T11:40:18.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:18.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836500 2026-02-17T11:40:18.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:18.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:40:18.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836500 2026-02-17T11:40:18.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:18.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836500 2026-02-17T11:40:18.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836500' 2026-02-17T11:40:18.476 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836500 2026-02-17T11:40:18.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:40:18.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836500 2026-02-17T11:40:18.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:40:19.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:40:19.632 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:40:19.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836500 -lt 21474836500 2026-02-17T11:40:19.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:19.790 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-210453397506 2026-02-17T11:40:19.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:19.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:40:19.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-210453397506 2026-02-17T11:40:19.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:19.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=210453397506 2026-02-17T11:40:19.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 210453397506' 2026-02-17T11:40:19.794 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 210453397506 2026-02-17T11:40:19.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:40:19.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 210453397506 -lt 210453397506 2026-02-17T11:40:19.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:19.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542161 2026-02-17T11:40:19.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:19.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:40:19.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542161 2026-02-17T11:40:19.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:19.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542161 2026-02-17T11:40:19.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542161' 2026-02-17T11:40:19.952 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542161 2026-02-17T11:40:19.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:40:20.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542161 -lt 60129542161 2026-02-17T11:40:20.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:20.113 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-188978561029 2026-02-17T11:40:20.113 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:20.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:40:20.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-188978561029 2026-02-17T11:40:20.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:20.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561029 2026-02-17T11:40:20.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 188978561029' 2026-02-17T11:40:20.117 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.3 seq 188978561029 2026-02-17T11:40:20.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:40:20.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561029 -lt 188978561029 2026-02-17T11:40:20.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:40:20.278 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:40:20.278 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:40:20.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:40:20.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:40:20.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:40:20.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:40:20.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:40:20.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:40:20.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:40:20.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:40:20.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:40:20.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:40:20.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:40:20.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:40:20.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:40:20.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:40:20.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:40:20.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-17T11:40:20.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:40:20.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:40:20.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:40:20.804 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:40:20.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=2.0 2026-02-17T11:40:20.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 2.0 2026-02-17T11:40:20.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T11:40:20.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T11:40:20.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:40:20.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:40:20.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:40:20.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:40:21.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:39:54.176074+0000 2026-02-17T11:40:21.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T11:40:21.244 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T11:40:21.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T11:39:54.176074+0000 2026-02-17T11:40:21.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:40:21.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:39:54.176074+0000 2026-02-17T11:40:21.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:40:21.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:40:21.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:40:21.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:40:21.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:40:21.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:40:21.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:40:21.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:40:21.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:39:54.176074+0000 '>' 2026-02-17T11:39:54.176074+0000 2026-02-17T11:40:21.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:40:22.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:40:22.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:40:22.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:40:22.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:40:22.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:40:22.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:40:22.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:40:22.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:39:54.176074+0000 '>' 2026-02-17T11:39:54.176074+0000 2026-02-17T11:40:22.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:40:23.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:40:23.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:40:23.557 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:40:23.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:40:23.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:40:23.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:40:23.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:40:23.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:39:54.176074+0000 '>' 2026-02-17T11:39:54.176074+0000 2026-02-17T11:40:23.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:40:24.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:40:24.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:40:24.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:40:24.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:40:24.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:40:24.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:40:24.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:40:24.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:39:54.176074+0000 '>' 2026-02-17T11:39:54.176074+0000 2026-02-17T11:40:24.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:40:25.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:40:25.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:40:25.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:40:25.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:40:25.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:40:25.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:40:25.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:40:26.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:39:54.176074+0000 '>' 2026-02-17T11:39:54.176074+0000 2026-02-17T11:40:26.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:40:27.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:40:27.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:40:27.019 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:40:27.019 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:40:27.019 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:40:27.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:40:27.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:40:27.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:40:21.592944+0000 '>' 2026-02-17T11:39:54.176074+0000 2026-02-17T11:40:27.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:40:27.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:40:27.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:40:27.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:40:27.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T11:40:27.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:40:27.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:40:27.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:40:27.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:40:27.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T11:40:27.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:40:27.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T11:40:27.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:40:27.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:40:27.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:40:27.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:40:27.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:40:28.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:40:28.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:40:28.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:40:28.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:40:28.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T11:40:28.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:40:28.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:40:28.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-17T11:40:28.792 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-17T11:40:28.793 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:40:28.793 INFO:tasks.workunit.client.0.trial128.stdout:hinfo_key 2026-02-17T11:40:28.793 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:40:29.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T11:40:29.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:40:29.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:40:29.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T11:40:29.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:40:29.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:40:29.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:40:29.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:40:29.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:40:29.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:40:29.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:40:29.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:40:29.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:40:29.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:40:29.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:40:29.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:40:29.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:40:29.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:40:29.343 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:40:29.343 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:40:29.344 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:40:29.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:40:29.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:40:29.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:40:29.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:40:29.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:40:29.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:40:29.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:40:29.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:40:29.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:40:29.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:40:29.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:40:29.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:40:29.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T11:40:29.347 INFO:tasks.workunit.client.0.trial128.stderr:start osd.3 2026-02-17T11:40:29.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:40:29.347 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T11:40:29.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T11:40:29.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:40:29.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:40:29.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:40:29.362 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:40:29.353+0000 7f617a4c68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:29.363 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:40:29.357+0000 7f617a4c68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:29.364 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:40:29.357+0000 7f617a4c68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:29.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T11:40:29.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:40:29.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:40:29.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:40:29.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:40:29.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:29.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:40:29.499 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:40:29.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:29.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:40:29.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:30.641 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:40:30.633+0000 7f617a4c68c0 -1 Falling back to public interface 2026-02-17T11:40:30.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:30.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:30.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:40:30.656 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:40:30.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:30.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:40:30.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:31.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:31.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:31.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:40:31.807 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:40:31.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:31.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:40:31.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:32.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:32.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:32.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:40:32.964 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:40:32.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:32.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:40:33.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:33.644 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:40:33.637+0000 7f617a4c68c0 -1 osd.3 51 log_to_monitors true 2026-02-17T11:40:34.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:34.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:34.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:40:34.115 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:40:34.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:34.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:40:34.261 INFO:tasks.workunit.client.0.trial128.stderr:osd.3 up in weight 1 up_from 55 up_thru 55 down_at 52 last_clean_interval [44,51) [v2:127.0.0.1:6826/2491455172,v1:127.0.0.1:6827/2491455172] [v2:127.0.0.1:6828/2491455172,v1:127.0.0.1:6829/2491455172] exists,up 2ce21662-caa1-40c6-be78-a6b6218bdef1 2026-02-17T11:40:34.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:40:34.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:40:34.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:40:34.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:40:34.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:40:34.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:40:34.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:40:34.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:40:34.263 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:40:34.263 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:40:34.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:40:34.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:40:34.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:40:34.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:40:34.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:40:34.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:40:34.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:40:34.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:40:34.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:40:34.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:40:34.522 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:40:34.523 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:40:34.523 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T11:40:34.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:40:34.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:34.523 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:40:34.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836504 2026-02-17T11:40:34.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836504 2026-02-17T11:40:34.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504' 2026-02-17T11:40:34.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:34.592 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:40:34.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=210453397510 2026-02-17T11:40:34.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 210453397510 2026-02-17T11:40:34.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-210453397510' 2026-02-17T11:40:34.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:34.664 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:40:34.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542165 2026-02-17T11:40:34.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542165 2026-02-17T11:40:34.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-210453397510 2-60129542165' 2026-02-17T11:40:34.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:34.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:40:34.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=236223201282 2026-02-17T11:40:34.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 236223201282 2026-02-17T11:40:34.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-210453397510 2-60129542165 3-236223201282' 2026-02-17T11:40:34.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:34.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836504 2026-02-17T11:40:34.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:34.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:40:34.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836504 2026-02-17T11:40:34.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:34.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836504 2026-02-17T11:40:34.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836504' 2026-02-17T11:40:34.813 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836504 2026-02-17T11:40:34.813 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:40:34.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836504 -lt 21474836504 2026-02-17T11:40:34.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:34.972 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-210453397510 2026-02-17T11:40:34.972 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:34.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:40:34.973 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-210453397510 2026-02-17T11:40:34.974 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:34.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=210453397510 2026-02-17T11:40:34.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 210453397510' 2026-02-17T11:40:34.975 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 210453397510 2026-02-17T11:40:34.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:40:35.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 210453397510 -lt 210453397510 2026-02-17T11:40:35.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:35.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542165 2026-02-17T11:40:35.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:35.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:40:35.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542165 2026-02-17T11:40:35.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:35.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542165 2026-02-17T11:40:35.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542165' 2026-02-17T11:40:35.137 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542165 2026-02-17T11:40:35.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:40:35.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542165 -lt 60129542165 2026-02-17T11:40:35.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:35.294 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-236223201282 2026-02-17T11:40:35.294 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:35.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:40:35.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-236223201282 2026-02-17T11:40:35.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:35.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=236223201282 2026-02-17T11:40:35.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 236223201282' 2026-02-17T11:40:35.298 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.3 seq 236223201282 2026-02-17T11:40:35.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:40:35.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 236223201282 -lt 236223201282 2026-02-17T11:40:35.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:40:35.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:40:35.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:40:35.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:40:35.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:40:35.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:40:35.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:40:35.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:40:35.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:40:35.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:40:35.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:40:35.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:40:35.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:40:35.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:40:35.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:40:35.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:40:35.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:40:35.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:40:35.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:40:36.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:40:36.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:262: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 1 2 2026-02-17T11:40:36.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:185: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-17T11:40:36.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:186: corrupt_and_repair_two: local poolname=ecpool 2026-02-17T11:40:36.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: corrupt_and_repair_two: local first=1 2026-02-17T11:40:36.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: corrupt_and_repair_two: local second=2 2026-02-17T11:40:36.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:193: corrupt_and_repair_two: pids= 2026-02-17T11:40:36.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:40:36.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:40:36.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:40:36.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 134596"' 2026-02-17T11:40:36.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 134596' 2026-02-17T11:40:36.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T11:40:36.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:40:36.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:40:36.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 134598"' 2026-02-17T11:40:36.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 134598' 2026-02-17T11:40:36.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:40:36.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: corrupt_and_repair_two: wait_background pids 2026-02-17T11:40:36.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 134596 134598' 2026-02-17T11:40:36.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T11:40:36.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:40:36.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 134596 2026-02-17T11:40:36.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/134599: /' 2026-02-17T11:40:36.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T11:40:36.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/134601: /' 2026-02-17T11:40:38.500 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:40:38.501 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:40:38.501 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T11:40:38.501 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:40:38.501 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T11:40:38.501 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:40:38.501 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:40:38.501 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T11:40:38.501 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:40:38.502 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T11:40:38.502 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:40:38.502 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:40:38.502 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:40:38.502 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:40:38.502 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:40:38.502 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:40:38.502 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T11:40:38.502 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:40:38.502 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:40:38.502 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T11:40:38.502 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:40:38.503 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:40:38.503 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING remove 2026-02-17T11:40:38.503 INFO:tasks.workunit.client.0.trial128.stderr:134601: remove 2#2:eb822e21:::SOMETHING:head# 2026-02-17T11:40:38.503 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T11:40:38.503 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:40:38.503 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:40:38.503 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T11:40:38.503 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:40:38.503 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:40:38.503 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:40:38.503 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:40:38.504 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:40:38.504 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:40:38.504 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:40:38.504 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:40:38.504 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:40:38.504 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:40:38.505 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:40:38.505 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:40:38.505 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:40:38.505 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:40:38.505 INFO:tasks.workunit.client.0.trial128.stderr:134601: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:40:38.505 INFO:tasks.workunit.client.0.trial128.stderr:134601: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:40:38.505 INFO:tasks.workunit.client.0.trial128.stderr:134601: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:40:38.505 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:40:38.505 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:40:38.505 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:40:38.505 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:40:38.505 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:40:38.506 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:40:38.506 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:40:38.506 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:40:38.506 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:40:38.506 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:40:38.506 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:40:38.506 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:40:38.506 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T11:40:38.506 INFO:tasks.workunit.client.0.trial128.stderr:134601: start osd.2 2026-02-17T11:40:38.506 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:40:38.759 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:40:38.760 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: start osd.1 2026-02-17T11:40:38.763 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:40:43.063 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:40:43.063 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:40:43.063 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:40:43.063 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:40:43.063 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:40:43.063 INFO:tasks.workunit.client.0.trial128.stderr:134599: 2026-02-17T11:40:38.773+0000 7f85f10228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:43.063 INFO:tasks.workunit.client.0.trial128.stderr:134599: 2026-02-17T11:40:38.773+0000 7f85f10228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:43.063 INFO:tasks.workunit.client.0.trial128.stderr:134599: 2026-02-17T11:40:38.773+0000 7f85f10228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:43.064 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:40:43.064 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:40:43.064 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:40:43.064 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:40:43.064 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:40:43.064 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:43.064 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:40:43.064 INFO:tasks.workunit.client.0.trial128.stderr:134599: 0 2026-02-17T11:40:43.064 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:43.064 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:40:43.064 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:43.064 INFO:tasks.workunit.client.0.trial128.stderr:134599: 2026-02-17T11:40:40.049+0000 7f85f10228c0 -1 Falling back to public interface 2026-02-17T11:40:43.064 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:43.064 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:43.064 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:40:43.065 INFO:tasks.workunit.client.0.trial128.stderr:134599: 1 2026-02-17T11:40:43.065 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:43.065 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:40:43.065 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:43.065 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:43.065 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:43.065 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:40:43.065 INFO:tasks.workunit.client.0.trial128.stderr:134599: 2 2026-02-17T11:40:43.065 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:43.065 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:40:43.065 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:43.065 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:43.065 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:43.065 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:40:43.066 INFO:tasks.workunit.client.0.trial128.stderr:134599: 3 2026-02-17T11:40:43.066 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:43.066 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:40:43.066 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:43.141 INFO:tasks.workunit.client.0.trial128.stderr:134599: 2026-02-17T11:40:43.053+0000 7f85f10228c0 134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T11:40:43.141 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T11:40:43.141 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:40:43.141 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:40:43.141 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:40:43.141 INFO:tasks.workunit.client.0.trial128.stderr:134601: 2026-02-17T11:40:38.517+0000 7f77bbbdd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:43.141 INFO:tasks.workunit.client.0.trial128.stderr:134601: 2026-02-17T11:40:38.517+0000 7f77bbbdd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:43.141 INFO:tasks.workunit.client.0.trial128.stderr:134601: 2026-02-17T11:40:38.517+0000 7f77bbbdd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:43.141 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T11:40:43.141 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:40:43.141 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:40:43.141 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:40:43.141 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:40:43.141 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:43.141 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:40:43.141 INFO:tasks.workunit.client.0.trial128.stderr:134601: 0 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: 1 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: 2026-02-17T11:40:40.081+0000 7f77bbbdd8c0 -1 Falling back to public interface 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: 2 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: 3 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:40:43.142 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:43.770 INFO:tasks.workunit.client.0.trial128.stderr:134601: 2026-02-17T11:40:43.133+0000 7f77bbbdd8c0 -1 osd.2 56 log_to_monitors true 2026-02-17T11:40:43.770 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:43.770 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:43.770 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:40:43.770 INFO:tasks.workunit.client.0.trial128.stderr:134601: 4 2026-02-17T11:40:43.770 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:43.770 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:40:43.770 INFO:tasks.workunit.client.0.trial128.stderr:134601: osd.2 up in weight 1 up_from 61 up_thru 46 down_at 57 last_clean_interval [14,56) [v2:127.0.0.1:6818/2253231270,v1:127.0.0.1:6819/2253231270] [v2:127.0.0.1:6820/2253231270,v1:127.0.0.1:6821/2253231270] exists,up 56d7ecf7-cec9-44c9-807a-2f5f5432a860 2026-02-17T11:40:43.770 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:40:43.770 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:40:43.770 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:40:43.770 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: 1 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: 2 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: 3' 2026-02-17T11:40:43.771 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:40:43.772 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:43.772 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:40:43.772 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836507 2026-02-17T11:40:43.772 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836507 2026-02-17T11:40:43.772 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507' 2026-02-17T11:40:43.772 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:44.022 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-hel-1 osd.1 56 log_to_monitors true 2026-02-17T11:40:44.022 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:44.022 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:44.022 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:40:44.022 INFO:tasks.workunit.client.0.trial128.stderr:134599: 4 2026-02-17T11:40:44.022 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: osd.1 up in weight 1 up_from 60 up_thru 61 down_at 57 last_clean_interval [49,56) [v2:127.0.0.1:6810/193849614,v1:127.0.0.1:6811/193849614] [v2:127.0.0.1:6812/193849614,v1:127.0.0.1:6813/193849614] exists,up 2f7d8768-732b-4677-9611-0afb8ab0865e 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: 1 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: 2 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: 3' 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:40:44.023 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:44.024 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:40:44.024 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836508 2026-02-17T11:40:44.024 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836508 2026-02-17T11:40:44.024 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508' 2026-02-17T11:40:44.024 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:45.300 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helperspers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:40:45.300 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=257698037762 2026-02-17T11:40:45.300 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 257698037762 2026-02-17T11:40:45.301 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-257698037762' 2026-02-17T11:40:45.301 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:45.301 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:40:45.301 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=261993005058 2026-02-17T11:40:45.301 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 261993005058 2026-02-17T11:40:45.301 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-257698037762 2-261993005058' 2026-02-17T11:40:45.301 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:45.301 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:40:45.301 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=236223201285 2026-02-17T11:40:45.301 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 236223201285 2026-02-17T11:40:45.302 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-257698037762 2-261993005058 3-236223201285' 2026-02-17T11:40:45.302 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:45.302 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836507 2026-02-17T11:40:45.302 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:45.302 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:40:45.302 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836507 2026-02-17T11:40:45.302 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:45.302 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836507 2026-02-17T11:40:45.302 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836507' 2026-02-17T11:40:45.302 INFO:tasks.workunit.client.0.trial128.stderr:134601: waiting osd.0 seq 21474836507 2026-02-17T11:40:45.302 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:40:45.303 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836506 -lt 21474836507 2026-02-17T11:40:45.303 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:40:45.303 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:40:45.303 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:40:45.303 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836508 -lt 21474836507 2026-02-17T11:40:45.303 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:45.303 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-257698037762 2026-02-17T11:40:45.303 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:45.303 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:40:45.303 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-257698037762 2026-02-17T11:40:45.303 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:45.562 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/u.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:40:45.562 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=257698037763 2026-02-17T11:40:45.562 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 257698037763 2026-02-17T11:40:45.562 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-257698037763' 2026-02-17T11:40:45.563 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:45.563 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:40:45.563 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=261993005059 2026-02-17T11:40:45.563 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 261993005059 2026-02-17T11:40:45.563 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-257698037763 2-261993005059' 2026-02-17T11:40:45.563 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:45.563 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:40:45.563 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=236223201286 2026-02-17T11:40:45.563 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 236223201286 2026-02-17T11:40:45.563 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-257698037763 2-261993005059 3-236223201286' 2026-02-17T11:40:45.564 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:45.564 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836508 2026-02-17T11:40:45.564 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:45.564 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:40:45.564 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836508 2026-02-17T11:40:45.564 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:45.564 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836508 2026-02-17T11:40:45.564 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836508' 2026-02-17T11:40:45.564 INFO:tasks.workunit.client.0.trial128.stderr:134599: waiting osd.0 seq 21474836508 2026-02-17T11:40:45.564 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:40:45.565 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836506 -lt 21474836508 2026-02-17T11:40:45.565 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:40:45.565 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:40:45.565 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:40:45.565 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836508 -lt 21474836508 2026-02-17T11:40:45.565 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:45.565 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-257698037763 2026-02-17T11:40:45.565 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:45.565 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:40:45.565 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-257698037763 2026-02-17T11:40:45.566 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:45.976 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntbuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=257698037762 2026-02-17T11:40:45.977 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 257698037762' 2026-02-17T11:40:45.977 INFO:tasks.workunit.client.0.trial128.stderr:134601: waiting osd.1 seq 257698037762 2026-02-17T11:40:45.977 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:40:45.977 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 257698037763 -lt 257698037762 2026-02-17T11:40:45.977 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:45.977 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-261993005058 2026-02-17T11:40:45.977 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:45.977 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:40:45.977 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-261993005058 2026-02-17T11:40:45.977 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:45.978 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=261993005058 2026-02-17T11:40:45.978 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 261993005058' 2026-02-17T11:40:45.978 INFO:tasks.workunit.client.0.trial128.stderr:134601: waiting osd.2 seq 261993005058 2026-02-17T11:40:45.978 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:40:45.978 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 261993005059 -lt 261993005058 2026-02-17T11:40:45.978 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:45.978 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-236223201285 2026-02-17T11:40:45.978 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:45.978 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:40:45.978 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-236223201285 2026-02-17T11:40:45.978 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:45.978 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=236223201285 2026-02-17T11:40:45.979 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 236223201285' 2026-02-17T11:40:45.979 INFO:tasks.workunit.client.0.trial128.stderr:134601: waiting osd.3 seq 236223201285 2026-02-17T11:40:45.979 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:40:45.979 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 236223201286 -lt 236223201285 2026-02-17T11:40:45.979 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:40:45.979 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:40:45.979 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:40:45.979 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:40:45.979 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:40:45.979 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:40:45.979 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:40:45.979 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:40:46.226 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("u/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=257698037763 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 257698037763' 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: waiting osd.1 seq 257698037763 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 257698037763 -lt 257698037763 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-261993005059 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-261993005059 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=261993005059 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 261993005059' 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: waiting osd.2 seq 261993005059 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 261993005059 -lt 261993005059 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-236223201286 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-236223201286 2026-02-17T11:40:46.227 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:46.228 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=236223201286 2026-02-17T11:40:46.228 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 236223201286' 2026-02-17T11:40:46.228 INFO:tasks.workunit.client.0.trial128.stderr:134599: waiting osd.3 seq 236223201286 2026-02-17T11:40:46.228 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:40:46.228 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 236223201286 -lt 236223201286 2026-02-17T11:40:46.228 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:40:46.228 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:40:46.228 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:40:46.228 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:40:46.228 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:40:46.228 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:40:46.228 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:40:46.228 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:40:46.320 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stalstale") | not)' 2026-02-17T11:40:46.321 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:40:46.321 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:40:46.321 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:40:46.321 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:40:46.321 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:40:46.321 INFO:tasks.workunit.client.0.trial128.stderr:134601: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:40:46.321 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:40:46.321 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:40:46.322 INFO:tasks.workunit.client.0.trial128.stderr:134601: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:40:46.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:40:46.570 INFO:tasks.workunit.client.0.trial128.stderr:e") | not)' 2026-02-17T11:40:46.570 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:40:46.570 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:40:46.570 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:40:46.570 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:40:46.570 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:40:46.570 INFO:tasks.workunit.client.0.trial128.stderr:134599: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:40:46.571 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:40:46.571 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:40:46.571 INFO:tasks.workunit.client.0.trial128.stderr:134599: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:40:46.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:40:46.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:40:46.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 134598 2026-02-17T11:40:46.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T11:40:46.573 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T11:40:46.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T11:40:46.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: corrupt_and_repair_two: return_code=0 2026-02-17T11:40:46.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:198: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T11:40:46.573 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-17T11:40:46.573 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:40:46.573 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:40:46.573 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:40:46.574 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:40:46.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: local pg=2.0 2026-02-17T11:40:46.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:204: corrupt_and_repair_two: repair 2.0 2026-02-17T11:40:46.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T11:40:46.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T11:40:46.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:40:46.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:40:46.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:40:46.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:40:46.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:40:21.592944+0000 2026-02-17T11:40:46.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T11:40:47.027 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T11:40:47.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T11:40:21.592944+0000 2026-02-17T11:40:47.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:40:47.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:40:21.592944+0000 2026-02-17T11:40:47.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:40:47.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:40:47.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:40:47.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:40:47.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:40:47.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:40:47.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:40:47.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:40:47.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:40:21.592944+0000 '>' 2026-02-17T11:40:21.592944+0000 2026-02-17T11:40:47.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:40:48.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:40:48.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:40:48.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:40:48.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:40:48.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:40:48.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:40:48.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:40:48.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:40:21.592944+0000 '>' 2026-02-17T11:40:21.592944+0000 2026-02-17T11:40:48.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:40:49.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:40:49.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:40:49.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:40:49.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:40:49.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:40:49.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:40:49.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:40:49.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:40:47.704424+0000 '>' 2026-02-17T11:40:21.592944+0000 2026-02-17T11:40:49.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:40:49.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: corrupt_and_repair_two: pids= 2026-02-17T11:40:49.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:40:49.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:40:49.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:40:49.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 137502"' 2026-02-17T11:40:49.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 137502' 2026-02-17T11:40:49.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-17T11:40:49.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:40:49.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:40:49.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 137504"' 2026-02-17T11:40:49.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:40:49.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 137504' 2026-02-17T11:40:49.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:211: corrupt_and_repair_two: wait_background pids 2026-02-17T11:40:49.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 137502 137504' 2026-02-17T11:40:49.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T11:40:49.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:40:49.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 137502 2026-02-17T11:40:49.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/137505: /' 2026-02-17T11:40:49.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-17T11:40:49.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/137508: /' 2026-02-17T11:40:51.672 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:40:51.672 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:40:51.673 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T11:40:51.673 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:40:51.673 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-17T11:40:51.673 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:40:51.673 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:40:51.673 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T11:40:51.673 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:40:51.673 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T11:40:51.673 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:40:51.673 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:40:51.673 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:40:51.673 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:40:51.674 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:40:51.674 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:40:51.674 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-17T11:40:51.674 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:40:51.674 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:40:51.674 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T11:40:51.674 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:40:51.674 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:40:51.674 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING list-attrs 2026-02-17T11:40:51.674 INFO:tasks.workunit.client.0.trial128.stderr:137508: _ 2026-02-17T11:40:51.674 INFO:tasks.workunit.client.0.trial128.stderr:137508: hinfo_key 2026-02-17T11:40:51.674 INFO:tasks.workunit.client.0.trial128.stderr:137508: snapset 2026-02-17T11:40:51.675 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T11:40:51.675 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:40:51.675 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:40:51.675 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T11:40:51.675 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:40:51.675 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:40:51.675 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:40:51.675 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:40:51.675 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helper137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:40:51.676 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:40:51.676 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:40:51.676 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:40:51.676 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:40:51.676 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:40:51.676 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:40:51.677 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:40:51.677 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:40:51.677 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:40:51.677 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:40:51.677 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:40:51.677 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:40:51.677 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:40:51.678 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:40:51.678 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:40:51.678 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:40:51.678 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:40:51.678 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:40:51.678 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:40:51.678 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:40:51.678 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:40:51.679 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T11:40:51.679 INFO:tasks.workunit.client.0.trial128.stderr:137505: _ 2026-02-17T11:40:51.679 INFO:tasks.workunit.client.0.trial128.stderr:137505: hinfo_key 2026-02-17T11:40:51.679 INFO:tasks.workunit.client.0.trial128.stderr:137505: snapset 2026-02-17T11:40:51.679 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:40:51.679 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:40:51.679 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:40:51.679 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:40:51.679 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:40:51.679 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:40:51.679 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:40:51.680 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:40:51.681 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:40:51.681 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:40:51.681 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:40:51.681 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:40:51.681 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:40:51.681 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:40:51.681 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:40:51.681 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:40:51.681 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:40:51.681 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:40:51.681 INFO:tasks.workunit.client.0.trial128.stderr:137508: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:40:51.682 INFO:tasks.workunit.client.0.trial128.stderr:137508: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:40:51.682 INFO:tasks.workunit.client.0.trial128.stderr:137508: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:40:51.682 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:40:51.682 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:40:51.682 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:40:51.682 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:40:51.682 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:40:51.682 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:40:51.682 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:40:51.682 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:40:51.682 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:40:51.682 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:40:51.682 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:40:51.683 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:40:51.683 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T11:40:51.683 INFO:tasks.workunit.client.0.trial128.stderr:137508: start osd.2 2026-02-17T11:40:51.683 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recs.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:40:51.683 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:40:51.683 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:40:51.683 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:40:51.683 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:40:51.683 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:40:51.683 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:40:51.683 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:40:51.683 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:40:51.683 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:40:51.684 INFO:tasks.workunit.client.0.trial128.stderr:137505: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:40:51.684 INFO:tasks.workunit.client.0.trial128.stderr:137505: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:40:51.684 INFO:tasks.workunit.client.0.trial128.stderr:137505: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:40:51.684 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:40:51.684 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:40:51.684 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:40:51.684 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:40:51.684 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:40:51.684 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:40:51.684 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:40:51.684 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:40:51.684 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:40:51.684 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:40:51.684 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:40:51.685 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:40:51.685 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:40:51.685 INFO:tasks.workunit.client.0.trial128.stderr:137505: start osd.1 2026-02-17T11:40:55.456 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:40:55.456 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T11:40:55.456 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T11:40:55.456 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:40:55.456 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:40:55.457 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:40:55.457 INFO:tasks.workunit.client.0.trial128.stderr:137508: 2026-02-17T11:40:51.685+0000 7f59919df8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:55.457 INFO:tasks.workunit.client.0.trial128.stderr:137508: 2026-02-17T11:40:51.685+0000 7f59919df8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:55.457 INFO:tasks.workunit.client.0.trial128.stderr:137508: 2026-02-17T11:40:51.685+0000 7f59919df8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:55.457 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T11:40:55.457 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:40:55.457 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:40:55.457 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:40:55.457 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:40:55.457 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:55.457 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:40:55.457 INFO:tasks.workunit.client.0.trial128.stderr:137508: 0 2026-02-17T11:40:55.458 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:55.458 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:40:55.458 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:55.458 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:55.458 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:55.458 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:40:55.458 INFO:tasks.workunit.client.0.trial128.stderr:137508: 1 2026-02-17T11:40:55.458 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:55.458 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:40:55.458 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:55.459 INFO:tasks.workunit.client.0.trial128.stderr:137508: 2026-02-17T11:40:53.157+0000 7f59919df8c0 -1 Falling back to public interface 2026-02-17T11:40:55.459 INFO:tasks.workunit.client.0.trial128.stderr:137508: 2026-02-17T11:40:53.157+0000 7f599099f640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6810/0: (98) Address already in use 2026-02-17T11:40:55.459 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:55.459 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:55.459 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:40:55.459 INFO:tasks.workunit.client.0.trial128.stderr:137508: 2 2026-02-17T11:40:55.459 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:55.459 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:40:55.459 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:55.459 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:55.459 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:55.459 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:40:55.460 INFO:tasks.workunit.client.0.trial128.stderr:137508: 3 2026-02-17T11:40:55.460 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:55.460 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:40:56.259 INFO:tasks.workunit.client.0.trial128.stderr:13750overy_ops 2026-02-17T11:40:56.259 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:40:56.260 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:40:56.260 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:40:56.260 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:40:56.260 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:40:56.260 INFO:tasks.workunit.client.0.trial128.stderr:137505: 2026-02-17T11:40:51.685+0000 7f22b52bc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:56.260 INFO:tasks.workunit.client.0.trial128.stderr:137505: 2026-02-17T11:40:51.685+0000 7f22b52bc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:56.261 INFO:tasks.workunit.client.0.trial128.stderr:137505: 2026-02-17T11:40:51.685+0000 7f22b52bc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:40:56.261 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:40:56.261 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:40:56.261 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:40:56.261 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:40:56.261 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:40:56.262 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:56.262 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:40:56.262 INFO:tasks.workunit.client.0.trial128.stderr:137505: 0 2026-02-17T11:40:56.262 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:56.262 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:40:56.262 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:56.263 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:56.263 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:56.263 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:40:56.263 INFO:tasks.workunit.client.0.trial128.stderr:137505: 1 2026-02-17T11:40:56.263 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:56.263 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:40:56.264 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:56.264 INFO:tasks.workunit.client.0.trial128.stderr:137505: 2026-02-17T11:40:53.157+0000 7f22b52bc8c0 -1 Falling back to public interface 2026-02-17T11:40:56.264 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:56.264 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:56.264 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:40:56.264 INFO:tasks.workunit.client.0.trial128.stderr:137505: 2 2026-02-17T11:40:56.264 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:56.265 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:40:56.265 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:56.265 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:56.265 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:56.265 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:40:56.265 INFO:tasks.workunit.client.0.trial128.stderr:137505: 3 2026-02-17T11:40:56.266 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:56.266 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:40:56.266 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:56.942 INFO:tasks.workunit.client.0.trial128.stderr:137505: 2026-02-17T11:40:56.249+0000 7f22b52bc8c0 -1 osd.1 62 log_to_monitors true 2026-02-17T11:40:56.942 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:56.942 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:56.942 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:40:56.942 INFO:tasks.workunit.client.0.trial128.stderr:137505: 4 2026-02-17T11:40:56.943 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:56.943 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:40:56.943 INFO:tasks.workunit.client.0.trial128.stderr:137505: osd.1 up in weight 1 up_from 66 up_thru 66 down_at 63 last_clean_interval [60,62) [v2:127.0.0.1:6810/1093168241,v1:127.0.0.1:6812/1093168241] [v2:127.0.0.1:6814/1093168241,v1:127.0.0.1:6816/1093168241] exists,up 2f7d8768-732b-4677-9611-0afb8ab0865e 2026-02-17T11:40:56.943 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:40:56.943 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:40:56.943 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:40:56.943 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:40:56.943 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:40:56.943 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:40:56.943 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:40:56.943 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:40:56.943 INFO:tasks.workunit.client.0.trial128.stderr:137505: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:40:56.943 INFO:tasks.workunit.client.0.trial128.stderr:137505: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:40:56.944 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:40:56.944 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:40:56.944 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:40:56.944 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:40:56.944 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:40:56.944 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:40:56.944 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:40:56.944 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:40:56.944 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:40:56.944 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:40:56.944 INFO:tasks.workunit.client.0.trial128.stderr:137505: 1 2026-02-17T11:40:56.944 INFO:tasks.workunit.client.0.trial128.stderr:137505: 2 2026-02-17T11:40:56.944 INFO:tasks.workunit.client.0.trial128.stderr:137505: 3' 2026-02-17T11:40:56.944 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:40:56.945 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:56.945 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:40:56.945 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836512 2026-02-17T11:40:56.945 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836512 2026-02-17T11:40:56.945 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836512' 2026-02-17T11:40:56.945 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:56.953 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalon8: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:40:56.953 INFO:tasks.workunit.client.0.trial128.stderr:137508: 2026-02-17T11:40:56.205+0000 7f59919df8c0 -1 osd.2 62 log_to_monitors true 2026-02-17T11:40:56.953 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:40:56.953 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:40:56.953 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:40:56.953 INFO:tasks.workunit.client.0.trial128.stderr:137508: 4 2026-02-17T11:40:56.953 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:40:56.953 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:40:56.953 INFO:tasks.workunit.client.0.trial128.stderr:137508: osd.2 up in weight 1 up_from 66 up_thru 46 down_at 63 last_clean_interval [61,62) [v2:127.0.0.1:6811/2472630395,v1:127.0.0.1:6813/2472630395] [v2:127.0.0.1:6815/2472630395,v1:127.0.0.1:6817/2472630395] exists,up 56d7ecf7-cec9-44c9-807a-2f5f5432a860 2026-02-17T11:40:56.953 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:40:56.954 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:40:56.954 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:40:56.954 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:40:56.954 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:40:56.954 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:40:56.954 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:40:56.954 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:40:56.954 INFO:tasks.workunit.client.0.trial128.stderr:137508: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:40:56.954 INFO:tasks.workunit.client.0.trial128.stderr:137508: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:40:56.954 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:40:56.954 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:40:56.954 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:40:56.954 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:40:56.955 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:40:56.955 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:40:56.955 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:40:56.955 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:40:56.955 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:40:56.955 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:40:56.955 INFO:tasks.workunit.client.0.trial128.stderr:137508: 1 2026-02-17T11:40:56.955 INFO:tasks.workunit.client.0.trial128.stderr:137508: 2 2026-02-17T11:40:56.955 INFO:tasks.workunit.client.0.trial128.stderr:137508: 3' 2026-02-17T11:40:56.955 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:40:56.955 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:56.955 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:40:56.955 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836513 2026-02-17T11:40:56.955 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836513 2026-02-17T11:40:56.956 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513' 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=283467841538 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 283467841538 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836512 1-283467841538' 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=283467841538 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 283467841538 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836512 1-283467841538 2-283467841538' 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=236223201289 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 236223201289 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836512 1-283467841538 2-283467841538 3-236223201289' 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836512 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836512 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836512 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836512' 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: waiting osd.0 seq 21474836512 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836511 -lt 21474836512 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:40:59.626 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:40:59.627 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:40:59.627 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836511 -lt 21474836512 2026-02-17T11:40:59.627 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:40:59.627 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:40:59.627 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:40:59.627 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836513 -lt 21474836512 2026-02-17T11:40:59.627 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:59.629 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_statse.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:59.629 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:40:59.630 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=283467841539 2026-02-17T11:40:59.630 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 283467841539 2026-02-17T11:40:59.630 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-283467841539' 2026-02-17T11:40:59.630 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:59.630 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:40:59.630 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=283467841539 2026-02-17T11:40:59.630 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 283467841539 2026-02-17T11:40:59.630 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-283467841539 2-283467841539' 2026-02-17T11:40:59.630 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:40:59.630 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:40:59.630 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=236223201290 2026-02-17T11:40:59.630 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 236223201290 2026-02-17T11:40:59.631 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-283467841539 2-283467841539 3-236223201290' 2026-02-17T11:40:59.631 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:40:59.631 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836513 2026-02-17T11:40:59.631 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:40:59.631 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:40:59.631 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836513 2026-02-17T11:40:59.631 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:40:59.631 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836513 2026-02-17T11:40:59.631 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836513' 2026-02-17T11:40:59.631 INFO:tasks.workunit.client.0.trial128.stderr:137508: waiting osd.0 seq 21474836513 2026-02-17T11:40:59.631 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:40:59.631 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836511 -lt 21474836513 2026-02-17T11:40:59.632 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:40:59.632 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:40:59.632 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:40:59.632 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836511 -lt 21474836513 2026-02-17T11:40:59.632 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:40:59.632 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:40:59.632 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:40:59.632 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836513 -lt 21474836513 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-283467841539 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-283467841539 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=283467841539 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 283467841539' 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: waiting osd.1 seq 283467841539 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 283467841539 -lt 283467841539 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-283467841539 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-283467841539 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=283467841539 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 283467841539' 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: waiting osd.2 seq 283467841539 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:41:00.285 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 283467841539 -lt 283467841539 2026-02-17T11:41:00.286 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:00.286 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-236223201290 2026-02-17T11:41:00.286 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:00.286 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:41:00.286 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-236223201290 2026-02-17T11:41:00.286 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:00.286 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=236223201290 2026-02-17T11:41:00.286 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 236223201290' 2026-02-17T11:41:00.286 INFO:tasks.workunit.client.0.trial128.stderr:137508: waiting osd.3 seq 236223201290 2026-02-17T11:41:00.286 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:41:00.286 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 236223201291 -lt 236223201290 2026-02-17T11:41:00.286 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:41:00.286 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:41:00.286 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:41:00.286 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:41:00.304 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cepht: echo 1-283467841538 2026-02-17T11:41:00.304 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:00.304 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:41:00.304 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-283467841538 2026-02-17T11:41:00.304 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:00.304 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=283467841538 2026-02-17T11:41:00.304 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 283467841538' 2026-02-17T11:41:00.304 INFO:tasks.workunit.client.0.trial128.stderr:137505: waiting osd.1 seq 283467841538 2026-02-17T11:41:00.304 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:41:00.304 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 283467841539 -lt 283467841538 2026-02-17T11:41:00.305 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:00.305 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-283467841538 2026-02-17T11:41:00.305 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:00.305 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:41:00.305 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-283467841538 2026-02-17T11:41:00.305 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:00.305 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=283467841538 2026-02-17T11:41:00.305 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 283467841538' 2026-02-17T11:41:00.305 INFO:tasks.workunit.client.0.trial128.stderr:137505: waiting osd.2 seq 283467841538 2026-02-17T11:41:00.305 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:41:00.305 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 283467841539 -lt 283467841538 2026-02-17T11:41:00.305 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:00.306 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-236223201289 2026-02-17T11:41:00.306 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:00.306 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:41:00.306 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-236223201289 2026-02-17T11:41:00.306 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:00.306 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=236223201289 2026-02-17T11:41:00.306 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 236223201289' 2026-02-17T11:41:00.306 INFO:tasks.workunit.client.0.trial128.stderr:137505: waiting osd.3 seq 236223201289 2026-02-17T11:41:00.306 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:41:00.306 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 236223201291 -lt 236223201289 2026-02-17T11:41:00.306 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:41:00.306 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:41:00.306 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:41:00.307 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:41:00.307 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:41:00.619 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/cest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:41:00.619 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:41:00.619 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:41:00.620 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:41:00.620 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:41:00.620 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:41:00.620 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:41:00.620 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:41:00.620 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:41:00.620 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:41:00.620 INFO:tasks.workunit.client.0.trial128.stderr:137508: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:41:00.620 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:41:00.620 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:41:00.620 INFO:tasks.workunit.client.0.trial128.stderr:137508: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:41:00.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:41:00.646 INFO:tasks.workunit.client.0.trial128.stderr:eph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:41:00.646 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:41:00.646 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:41:00.647 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:41:00.647 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:41:00.647 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:41:00.647 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:41:00.647 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:41:00.647 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:41:00.647 INFO:tasks.workunit.client.0.trial128.stderr:137505: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:41:00.647 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:41:00.647 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:41:00.647 INFO:tasks.workunit.client.0.trial128.stderr:137505: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:41:00.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:41:00.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:41:00.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 137504 2026-02-17T11:41:00.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T11:41:00.649 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T11:41:00.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T11:41:00.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:212: corrupt_and_repair_two: return_code=0 2026-02-17T11:41:00.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:213: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T11:41:00.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:41:00.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:41:00.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 3 1 2026-02-17T11:41:00.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:185: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-17T11:41:00.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:186: corrupt_and_repair_two: local poolname=ecpool 2026-02-17T11:41:00.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: corrupt_and_repair_two: local first=3 2026-02-17T11:41:00.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: corrupt_and_repair_two: local second=1 2026-02-17T11:41:00.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:193: corrupt_and_repair_two: pids= 2026-02-17T11:41:00.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:41:00.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:41:00.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:41:00.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 139965"' 2026-02-17T11:41:00.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 139965' 2026-02-17T11:41:00.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:41:00.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:41:00.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:41:00.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 139967"' 2026-02-17T11:41:00.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 139967' 2026-02-17T11:41:00.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: corrupt_and_repair_two: wait_background pids 2026-02-17T11:41:00.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:41:00.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 139965 139967' 2026-02-17T11:41:00.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T11:41:00.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:41:00.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 139965 2026-02-17T11:41:00.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/139968: /' 2026-02-17T11:41:00.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:41:00.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/139970: /' 2026-02-17T11:41:03.153 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:41:03.153 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:41:03.154 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T11:41:03.154 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:41:03.154 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:41:03.154 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:41:03.154 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:41:03.154 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T11:41:03.154 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:41:03.154 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T11:41:03.154 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:41:03.154 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:41:03.154 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:41:03.155 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:41:03.155 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:41:03.155 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:41:03.155 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:41:03.155 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:41:03.155 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:41:03.155 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T11:41:03.155 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:41:03.155 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:41:03.155 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T11:41:03.155 INFO:tasks.workunit.client.0.trial128.stderr:139968: remove 0#2:eb822e21:::SOMETHING:head# 2026-02-17T11:41:03.155 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T11:41:03.156 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:41:03.156 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:41:03.156 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T11:41:03.156 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:41:03.156 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:41:03.156 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:41:03.156 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:41:03.157 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:41:03.157 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:41:03.157 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:41:03.157 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:41:03.157 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:41:03.157 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:41:03.158 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:41:03.158 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:41:03.158 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:41:03.158 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:41:03.158 INFO:tasks.workunit.client.0.trial128.stderr:139968: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:41:03.158 INFO:tasks.workunit.client.0.trial128.stderr:139968: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:41:03.158 INFO:tasks.workunit.client.0.trial128.stderr:139968: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:41:03.158 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:03.158 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:03.158 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:41:03.159 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:41:03.159 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:41:03.159 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:41:03.159 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:41:03.159 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:41:03.159 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:41:03.159 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:41:03.159 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:41:03.159 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:41:03.159 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T11:41:03.159 INFO:tasks.workunit.client.0.trial128.stderr:139968: start osd.3 2026-02-17T11:41:03.160 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:41:03.163 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:41:03.164 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:41:03.164 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:41:03.164 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:41:03.164 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:41:03.164 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:41:03.164 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:41:03.164 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:41:03.164 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:41:03.164 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:41:03.164 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:41:03.164 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:41:03.165 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:41:03.165 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:41:03.165 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:41:03.165 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:41:03.165 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:41:03.165 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:41:03.165 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:41:03.165 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:41:03.165 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:41:03.165 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:41:03.165 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T11:41:03.165 INFO:tasks.workunit.client.0.trial128.stderr:139970: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T11:41:03.165 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:41:03.166 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:41:03.166 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:41:03.166 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:41:03.166 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:41:03.166 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:41:03.166 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:41:03.166 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:41:03.167 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:41:03.167 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:41:03.167 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:41:03.167 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:41:03.167 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:41:03.168 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:41:03.168 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:41:03.168 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:41:03.168 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:41:03.168 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:41:03.168 INFO:tasks.workunit.client.0.trial128.stderr:139970: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:41:03.168 INFO:tasks.workunit.client.0.trial128.stderr:139970: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:41:03.168 INFO:tasks.workunit.client.0.trial128.stderr:139970: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:41:03.168 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:03.168 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:03.168 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:41:03.168 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:41:03.169 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:41:03.169 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:41:03.169 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:41:03.169 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:41:03.169 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:41:03.169 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:41:03.169 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:41:03.169 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:41:03.169 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:41:03.170 INFO:tasks.workunit.client.0.trial128.stderr:139970: start osd.1 2026-02-17T11:41:03.170 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:41:07.498 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:41:07.498 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:41:07.498 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:41:07.498 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:41:07.498 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:41:07.498 INFO:tasks.workunit.client.0.trial128.stderr:139970: 2026-02-17T11:41:03.172+0000 7f52d68de8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:07.499 INFO:tasks.workunit.client.0.trial128.stderr:139970: 2026-02-17T11:41:03.176+0000 7f52d68de8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:07.499 INFO:tasks.workunit.client.0.trial128.stderr:139970: 2026-02-17T11:41:03.176+0000 7f52d68de8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:07.499 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:41:07.499 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:41:07.499 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:41:07.499 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:41:07.499 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:41:07.499 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:07.499 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:41:07.499 INFO:tasks.workunit.client.0.trial128.stderr:139970: 0 2026-02-17T11:41:07.499 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:07.499 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:41:07.499 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:07.500 INFO:tasks.workunit.client.0.trial128.stderr:139970: 2026-02-17T11:41:04.436+0000 7f52d68de8c0 -1 Falling back to public interface 2026-02-17T11:41:07.500 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:07.500 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:07.500 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:41:07.500 INFO:tasks.workunit.client.0.trial128.stderr:139970: 1 2026-02-17T11:41:07.500 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:07.500 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:41:07.500 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:07.500 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:07.500 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:07.500 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:41:07.500 INFO:tasks.workunit.client.0.trial128.stderr:139970: 2 2026-02-17T11:41:07.501 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:07.501 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:41:07.501 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:07.501 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:07.501 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:07.501 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:41:07.501 INFO:tasks.workunit.client.0.trial128.stderr:139970: 3 2026-02-17T11:41:07.501 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:07.501 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:41:07.501 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:07.723 INFO:tasks.workunit.client.0.trial128.stderr:139970: 2026-02-17T11:41:07.488+0000 7f52d68de8c0 139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T11:41:07.724 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T11:41:07.724 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:41:07.724 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:41:07.724 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:41:07.724 INFO:tasks.workunit.client.0.trial128.stderr:139968: 2026-02-17T11:41:03.172+0000 7fec5ae668c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:07.724 INFO:tasks.workunit.client.0.trial128.stderr:139968: 2026-02-17T11:41:03.176+0000 7fec5ae668c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:07.724 INFO:tasks.workunit.client.0.trial128.stderr:139968: 2026-02-17T11:41:03.176+0000 7fec5ae668c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:07.724 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T11:41:07.724 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:41:07.724 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:41:07.724 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:41:07.724 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:41:07.724 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: 0 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: 1 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: 2026-02-17T11:41:04.668+0000 7fec5ae668c0 -1 Falling back to public interface 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: 2 2026-02-17T11:41:07.725 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:07.726 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:41:07.726 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:07.726 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:07.726 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:07.726 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:41:07.726 INFO:tasks.workunit.client.0.trial128.stderr:139968: 3 2026-02-17T11:41:07.726 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:07.726 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:41:07.726 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:08.414 INFO:tasks.workunit.client.0.trial128.stderr:139968: 2026-02-17T11:41:07.716+0000 7fec5ae668c0 -1 osd.3 68 log_to_monitors true 2026-02-17T11:41:08.414 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:08.414 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:08.414 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:41:08.414 INFO:tasks.workunit.client.0.trial128.stderr:139968: 4 2026-02-17T11:41:08.414 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:08.414 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:41:08.414 INFO:tasks.workunit.client.0.trial128.stderr:139968: osd.3 up in weight 1 up_from 74 up_thru 74 down_at 69 last_clean_interval [55,68) [v2:127.0.0.1:6826/4128843934,v1:127.0.0.1:6827/4128843934] [v2:127.0.0.1:6828/4128843934,v1:127.0.0.1:6829/4128843934] exists,up 2ce21662-caa1-40c6-be78-a6b6218bdef1 2026-02-17T11:41:08.414 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:41:08.414 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:41:08.414 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:41:08.414 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:41:08.414 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:41:08.414 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:41:08.415 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:41:08.415 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:41:08.415 INFO:tasks.workunit.client.0.trial128.stderr:139968: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:41:08.415 INFO:tasks.workunit.client.0.trial128.stderr:139968: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:41:08.415 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:41:08.415 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:41:08.415 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:41:08.415 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:41:08.415 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:41:08.415 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:41:08.415 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:41:08.415 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:41:08.416 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:41:08.416 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:41:08.416 INFO:tasks.workunit.client.0.trial128.stderr:139968: 1 2026-02-17T11:41:08.416 INFO:tasks.workunit.client.0.trial128.stderr:139968: 2 2026-02-17T11:41:08.416 INFO:tasks.workunit.client.0.trial128.stderr:139968: 3' 2026-02-17T11:41:08.416 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:41:08.416 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:41:08.416 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:41:08.416 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836516 2026-02-17T11:41:08.416 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836516 2026-02-17T11:41:08.416 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836516' 2026-02-17T11:41:08.416 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:41:08.440 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-hel-1 osd.1 68 log_to_monitors true 2026-02-17T11:41:08.440 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:08.441 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:08.441 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:41:08.441 INFO:tasks.workunit.client.0.trial128.stderr:139970: 4 2026-02-17T11:41:08.441 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:08.441 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:41:08.441 INFO:tasks.workunit.client.0.trial128.stderr:139970: osd.1 up in weight 1 up_from 72 up_thru 72 down_at 69 last_clean_interval [66,68) [v2:127.0.0.1:6810/4025082010,v1:127.0.0.1:6812/4025082010] [v2:127.0.0.1:6814/4025082010,v1:127.0.0.1:6816/4025082010] exists,up 2f7d8768-732b-4677-9611-0afb8ab0865e 2026-02-17T11:41:08.441 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:41:08.441 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:41:08.441 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:41:08.441 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:41:08.441 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:41:08.442 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:41:08.442 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:41:08.442 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:41:08.442 INFO:tasks.workunit.client.0.trial128.stderr:139970: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:41:08.442 INFO:tasks.workunit.client.0.trial128.stderr:139970: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:41:08.442 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:41:08.442 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:41:08.442 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:41:08.442 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:41:08.442 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:41:08.442 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:41:08.442 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:41:08.443 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:41:08.443 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:41:08.443 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:41:08.443 INFO:tasks.workunit.client.0.trial128.stderr:139970: 1 2026-02-17T11:41:08.443 INFO:tasks.workunit.client.0.trial128.stderr:139970: 2 2026-02-17T11:41:08.443 INFO:tasks.workunit.client.0.trial128.stderr:139970: 3' 2026-02-17T11:41:08.443 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:41:08.443 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:41:08.443 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:41:08.443 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836517 2026-02-17T11:41:08.443 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836517 2026-02-17T11:41:08.443 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517' 2026-02-17T11:41:08.444 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:41:09.955 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:41:09.955 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=309237645314 2026-02-17T11:41:09.956 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 309237645314 2026-02-17T11:41:09.956 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836516 1-309237645314' 2026-02-17T11:41:09.956 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:41:09.956 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:41:09.956 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=283467841542 2026-02-17T11:41:09.956 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 283467841542 2026-02-17T11:41:09.956 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836516 1-309237645314 2-283467841542' 2026-02-17T11:41:09.956 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:41:09.956 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:41:09.956 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=317827579906 2026-02-17T11:41:09.956 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 317827579906 2026-02-17T11:41:09.957 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836516 1-309237645314 2-283467841542 3-317827579906' 2026-02-17T11:41:09.957 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:09.957 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836516 2026-02-17T11:41:09.957 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:09.957 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:41:09.957 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836516 2026-02-17T11:41:09.957 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:09.957 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836516 2026-02-17T11:41:09.957 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836516' 2026-02-17T11:41:09.957 INFO:tasks.workunit.client.0.trial128.stderr:139968: waiting osd.0 seq 21474836516 2026-02-17T11:41:09.957 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:41:09.957 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836515 -lt 21474836516 2026-02-17T11:41:09.958 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:41:09.958 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:41:09.958 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:41:09.958 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836517 -lt 21474836516 2026-02-17T11:41:09.958 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:09.958 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-309237645314 2026-02-17T11:41:09.958 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:09.958 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:41:09.958 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-309237645314 2026-02-17T11:41:09.958 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:09.965 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/upers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:41:09.966 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=309237645315 2026-02-17T11:41:09.966 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 309237645315 2026-02-17T11:41:09.966 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-309237645315' 2026-02-17T11:41:09.966 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:41:09.966 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:41:09.966 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=283467841543 2026-02-17T11:41:09.966 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 283467841543 2026-02-17T11:41:09.966 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-309237645315 2-283467841543' 2026-02-17T11:41:09.966 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:41:09.966 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:41:09.967 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=317827579907 2026-02-17T11:41:09.967 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 317827579907 2026-02-17T11:41:09.967 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-309237645315 2-283467841543 3-317827579907' 2026-02-17T11:41:09.967 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:09.967 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836517 2026-02-17T11:41:09.967 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:09.967 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:41:09.967 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836517 2026-02-17T11:41:09.967 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:09.967 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836517 2026-02-17T11:41:09.967 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836517' 2026-02-17T11:41:09.967 INFO:tasks.workunit.client.0.trial128.stderr:139970: waiting osd.0 seq 21474836517 2026-02-17T11:41:09.967 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:41:09.968 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836515 -lt 21474836517 2026-02-17T11:41:09.968 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:41:09.968 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:41:09.968 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:41:09.968 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836517 -lt 21474836517 2026-02-17T11:41:09.968 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:09.968 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-309237645315 2026-02-17T11:41:09.968 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:09.968 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:41:09.968 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-309237645315 2026-02-17T11:41:09.968 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:10.603 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=309237645314 2026-02-17T11:41:10.603 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 309237645314' 2026-02-17T11:41:10.603 INFO:tasks.workunit.client.0.trial128.stderr:139968: waiting osd.1 seq 309237645314 2026-02-17T11:41:10.603 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:41:10.603 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 309237645315 -lt 309237645314 2026-02-17T11:41:10.603 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:10.603 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-283467841542 2026-02-17T11:41:10.603 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:10.603 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:41:10.603 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-283467841542 2026-02-17T11:41:10.603 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:10.603 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=283467841542 2026-02-17T11:41:10.603 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 283467841542' 2026-02-17T11:41:10.603 INFO:tasks.workunit.client.0.trial128.stderr:139968: waiting osd.2 seq 283467841542 2026-02-17T11:41:10.603 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:41:10.603 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 283467841543 -lt 283467841542 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-317827579906 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-317827579906 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=317827579906 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 317827579906' 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: waiting osd.3 seq 317827579906 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 317827579907 -lt 317827579906 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:41:10.604 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:41:10.623 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("buntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=309237645315 2026-02-17T11:41:10.623 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 309237645315' 2026-02-17T11:41:10.624 INFO:tasks.workunit.client.0.trial128.stderr:139970: waiting osd.1 seq 309237645315 2026-02-17T11:41:10.624 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:41:10.624 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 309237645315 -lt 309237645315 2026-02-17T11:41:10.624 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:10.624 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-283467841543 2026-02-17T11:41:10.624 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:10.624 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:41:10.624 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-283467841543 2026-02-17T11:41:10.624 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:10.624 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=283467841543 2026-02-17T11:41:10.624 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 283467841543' 2026-02-17T11:41:10.625 INFO:tasks.workunit.client.0.trial128.stderr:139970: waiting osd.2 seq 283467841543 2026-02-17T11:41:10.625 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:41:10.625 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 283467841543 -lt 283467841543 2026-02-17T11:41:10.625 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:10.625 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-317827579907 2026-02-17T11:41:10.625 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:10.625 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:41:10.625 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-317827579907 2026-02-17T11:41:10.625 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:10.625 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=317827579907 2026-02-17T11:41:10.625 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 317827579907' 2026-02-17T11:41:10.625 INFO:tasks.workunit.client.0.trial128.stderr:139970: waiting osd.3 seq 317827579907 2026-02-17T11:41:10.625 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:41:10.626 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 317827579907 -lt 317827579907 2026-02-17T11:41:10.626 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:41:10.626 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:41:10.626 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:41:10.626 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:41:10.626 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:41:10.626 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:41:10.626 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:41:10.626 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:41:10.948 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:41:10.948 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:41:10.949 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:41:10.949 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:41:10.949 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:41:10.949 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:41:10.949 INFO:tasks.workunit.client.0.trial128.stderr:139968: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:41:10.949 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:41:10.949 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:41:10.949 INFO:tasks.workunit.client.0.trial128.stderr:139968: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:41:10.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:41:10.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:41:10.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 139967 2026-02-17T11:41:10.950 INFO:tasks.workunit.client.0.trial128.stderr:stale") | not)' 2026-02-17T11:41:10.950 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:41:10.950 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:41:10.950 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:41:10.950 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:41:10.950 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:41:10.950 INFO:tasks.workunit.client.0.trial128.stderr:139970: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:41:10.950 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:41:10.950 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:41:10.950 INFO:tasks.workunit.client.0.trial128.stderr:139970: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:41:10.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:41:10.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T11:41:10.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T11:41:10.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T11:41:10.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: corrupt_and_repair_two: return_code=0 2026-02-17T11:41:10.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:198: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T11:41:10.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-17T11:41:10.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:41:10.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:41:10.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:41:10.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:41:11.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: local pg=2.0 2026-02-17T11:41:11.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:204: corrupt_and_repair_two: repair 2.0 2026-02-17T11:41:11.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T11:41:11.110 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T11:41:11.110 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:41:11.110 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:41:11.110 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:41:11.110 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:41:11.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:40:47.704424+0000 2026-02-17T11:41:11.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T11:41:11.401 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T11:41:11.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T11:40:47.704424+0000 2026-02-17T11:41:11.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:41:11.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:40:47.704424+0000 2026-02-17T11:41:11.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:41:11.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:41:11.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:41:11.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:41:11.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:41:11.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:41:11.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:41:11.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:41:11.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:40:47.704424+0000 '>' 2026-02-17T11:40:47.704424+0000 2026-02-17T11:41:11.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:41:12.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:41:12.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:41:12.567 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:41:12.567 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:41:12.567 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:41:12.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:41:12.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:41:12.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:40:47.704424+0000 '>' 2026-02-17T11:40:47.704424+0000 2026-02-17T11:41:12.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:41:13.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:41:13.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:41:13.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:41:13.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:41:13.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:41:13.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:41:13.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:41:13.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:41:11.792066+0000 '>' 2026-02-17T11:40:47.704424+0000 2026-02-17T11:41:13.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:41:13.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: corrupt_and_repair_two: pids= 2026-02-17T11:41:13.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:41:13.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:41:13.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:41:13.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 142871"' 2026-02-17T11:41:13.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 142871' 2026-02-17T11:41:13.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:41:13.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:41:13.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:41:13.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 142873"' 2026-02-17T11:41:13.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:41:13.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 142873' 2026-02-17T11:41:13.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:211: corrupt_and_repair_two: wait_background pids 2026-02-17T11:41:13.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 142871 142873' 2026-02-17T11:41:13.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T11:41:13.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:41:13.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 142871 2026-02-17T11:41:13.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/142874: /' 2026-02-17T11:41:13.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:41:13.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/142877: /' 2026-02-17T11:41:16.040 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:41:16.040 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:41:16.040 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:41:16.040 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:41:16.040 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:41:16.040 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:41:16.040 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:41:16.040 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:41:16.040 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: _ 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: hinfo_key 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: snapset 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:41:16.041 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:41:16.042 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:41:16.044 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:41:16.044 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:41:16.044 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:41:16.044 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:41:16.044 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:41:16.044 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:41:16.044 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:41:16.044 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:41:16.044 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:41:16.044 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:41:16.044 INFO:tasks.workunit.client.0.trial128.stderr:142877: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:41:16.044 INFO:tasks.workunit.client.0.trial128.stderr:142877: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:41:16.044 INFO:tasks.workunit.client.0.trial128.stderr:142877: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:41:16.045 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:16.045 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:16.045 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:41:16.045 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:41:16.045 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:41:16.045 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:41:16.045 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:41:16.045 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:41:16.045 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:41:16.045 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:41:16.045 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:41:16.045 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:41:16.045 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:41:16.045 INFO:tasks.workunit.client.0.trial128.stderr:142877: start osd.1 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_rec142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:41:16.079 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T11:41:16.080 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:41:16.080 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:41:16.080 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-17T11:41:16.080 INFO:tasks.workunit.client.0.trial128.stderr:142874: Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-17T11:41:16.080 INFO:tasks.workunit.client.0.trial128.stderr:142874: _ 2026-02-17T11:41:16.080 INFO:tasks.workunit.client.0.trial128.stderr:142874: hinfo_key 2026-02-17T11:41:16.080 INFO:tasks.workunit.client.0.trial128.stderr:142874: snapset 2026-02-17T11:41:16.080 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T11:41:16.080 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:41:16.080 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:41:16.080 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T11:41:16.080 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:41:16.080 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:41:16.080 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:41:16.082 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:41:16.082 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:41:16.082 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:41:16.082 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:41:16.082 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:41:16.082 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:41:16.082 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:41:16.082 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:41:16.083 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:41:16.083 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:41:16.083 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:41:16.083 INFO:tasks.workunit.client.0.trial128.stderr:142874: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:41:16.083 INFO:tasks.workunit.client.0.trial128.stderr:142874: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:41:16.083 INFO:tasks.workunit.client.0.trial128.stderr:142874: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:41:16.083 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:16.083 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:16.083 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:41:16.083 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:41:16.083 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:41:16.083 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:41:16.084 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:41:16.084 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:41:16.084 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:41:16.084 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:41:16.084 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:41:16.084 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:41:16.084 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T11:41:16.084 INFO:tasks.workunit.client.0.trial128.stderr:142874: start osd.3 2026-02-17T11:41:19.836 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:41:19.837 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T11:41:19.837 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T11:41:19.837 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:41:19.837 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:41:19.837 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:41:19.837 INFO:tasks.workunit.client.0.trial128.stderr:142874: 2026-02-17T11:41:16.088+0000 7fe6d8f0f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:19.837 INFO:tasks.workunit.client.0.trial128.stderr:142874: 2026-02-17T11:41:16.088+0000 7fe6d8f0f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:19.837 INFO:tasks.workunit.client.0.trial128.stderr:142874: 2026-02-17T11:41:16.088+0000 7fe6d8f0f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:19.837 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T11:41:19.838 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:41:19.838 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:41:19.838 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:41:19.838 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:41:19.838 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:19.838 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:41:19.838 INFO:tasks.workunit.client.0.trial128.stderr:142874: 0 2026-02-17T11:41:19.838 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:19.838 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:41:19.838 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:19.838 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:19.838 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:19.839 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:41:19.839 INFO:tasks.workunit.client.0.trial128.stderr:142874: 1 2026-02-17T11:41:19.839 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:19.839 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:41:19.839 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:19.839 INFO:tasks.workunit.client.0.trial128.stderr:142874: 2026-02-17T11:41:17.608+0000 7fe6d8f0f8c0 -1 Falling back to public interface 2026-02-17T11:41:19.839 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:19.839 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:19.839 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:41:19.839 INFO:tasks.workunit.client.0.trial128.stderr:142874: 2 2026-02-17T11:41:19.839 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:19.839 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:41:19.840 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:19.840 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:19.840 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:19.840 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:41:19.840 INFO:tasks.workunit.client.0.trial128.stderr:142874: 3 2026-02-17T11:41:19.840 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:19.840 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:41:20.345 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.clientovery_ops 2026-02-17T11:41:20.345 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:41:20.346 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:41:20.346 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:41:20.346 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:41:20.346 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:41:20.346 INFO:tasks.workunit.client.0.trial128.stderr:142877: 2026-02-17T11:41:16.056+0000 7f491e8148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:20.346 INFO:tasks.workunit.client.0.trial128.stderr:142877: 2026-02-17T11:41:16.056+0000 7f491e8148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:20.346 INFO:tasks.workunit.client.0.trial128.stderr:142877: 2026-02-17T11:41:16.056+0000 7f491e8148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:20.346 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:41:20.346 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:41:20.347 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:41:20.347 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:41:20.347 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:41:20.347 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:20.347 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:41:20.347 INFO:tasks.workunit.client.0.trial128.stderr:142877: 0 2026-02-17T11:41:20.347 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:20.347 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:41:20.347 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:20.348 INFO:tasks.workunit.client.0.trial128.stderr:142877: 2026-02-17T11:41:17.320+0000 7f491e8148c0 -1 Falling back to public interface 2026-02-17T11:41:20.348 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:20.348 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:20.348 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:41:20.348 INFO:tasks.workunit.client.0.trial128.stderr:142877: 1 2026-02-17T11:41:20.348 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:20.348 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:41:20.348 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:20.348 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:20.348 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:20.348 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:41:20.348 INFO:tasks.workunit.client.0.trial128.stderr:142877: 2 2026-02-17T11:41:20.349 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:20.349 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:41:20.349 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:20.349 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:20.349 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:20.349 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:41:20.349 INFO:tasks.workunit.client.0.trial128.stderr:142877: 3 2026-02-17T11:41:20.349 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:20.349 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:41:20.349 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:21.306 INFO:tasks.workunit.client.0.trial128.stderr:142877: 2026-02-17T11:41:20.336+0000 7f491e8148c0 -1 osd.1 75 log_to_monitors true 2026-02-17T11:41:21.307 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:21.307 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:21.307 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:41:21.307 INFO:tasks.workunit.client.0.trial128.stderr:142877: 4 2026-02-17T11:41:21.307 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:21.307 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:41:21.307 INFO:tasks.workunit.client.0.trial128.stderr:142877: osd.1 up in weight 1 up_from 79 up_thru 79 down_at 76 last_clean_interval [72,75) [v2:127.0.0.1:6810/293037604,v1:127.0.0.1:6812/293037604] [v2:127.0.0.1:6814/293037604,v1:127.0.0.1:6816/293037604] exists,up 2f7d8768-732b-4677-9611-0afb8ab0865e 2026-02-17T11:41:21.307 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:41:21.307 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:41:21.307 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:41:21.307 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:41:21.307 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:41:21.307 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:41:21.308 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:41:21.308 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:41:21.308 INFO:tasks.workunit.client.0.trial128.stderr:142877: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:41:21.308 INFO:tasks.workunit.client.0.trial128.stderr:142877: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:41:21.308 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:41:21.308 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:41:21.308 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:41:21.308 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:41:21.308 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:41:21.308 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:41:21.308 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:41:21.308 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:41:21.308 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:41:21.308 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:41:21.308 INFO:tasks.workunit.client.0.trial128.stderr:142877: 1 2026-02-17T11:41:21.308 INFO:tasks.workunit.client.0.trial128.stderr:142877: 2 2026-02-17T11:41:21.309 INFO:tasks.workunit.client.0.trial128.stderr:142877: 3' 2026-02-17T11:41:21.309 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:41:21.309 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:41:21.309 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:41:21.309 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836521 2026-02-17T11:41:21.309 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836521 2026-02-17T11:41:21.309 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521' 2026-02-17T11:41:21.309 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:41:21.322 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ce.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:21.323 INFO:tasks.workunit.client.0.trial128.stderr:142874: 2026-02-17T11:41:20.700+0000 7fe6d8f0f8c0 -1 osd.3 75 log_to_monitors true 2026-02-17T11:41:21.323 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:21.323 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:21.323 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:41:21.323 INFO:tasks.workunit.client.0.trial128.stderr:142874: 4 2026-02-17T11:41:21.323 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:21.323 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:41:21.323 INFO:tasks.workunit.client.0.trial128.stderr:142874: osd.3 up in weight 1 up_from 83 up_thru 74 down_at 76 last_clean_interval [74,75) [v2:127.0.0.1:6826/2184282968,v1:127.0.0.1:6827/2184282968] [v2:127.0.0.1:6828/2184282968,v1:127.0.0.1:6829/2184282968] exists,up 2ce21662-caa1-40c6-be78-a6b6218bdef1 2026-02-17T11:41:21.323 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:41:21.323 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:41:21.323 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:41:21.323 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:41:21.323 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:41:21.324 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:41:21.324 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:41:21.324 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:41:21.324 INFO:tasks.workunit.client.0.trial128.stderr:142874: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:41:21.324 INFO:tasks.workunit.client.0.trial128.stderr:142874: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:41:21.324 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:41:21.324 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:41:21.324 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:41:21.324 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:41:21.324 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:41:21.324 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:41:21.324 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:41:21.324 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:41:21.324 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:41:21.324 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:41:21.324 INFO:tasks.workunit.client.0.trial128.stderr:142874: 1 2026-02-17T11:41:21.325 INFO:tasks.workunit.client.0.trial128.stderr:142874: 2 2026-02-17T11:41:21.325 INFO:tasks.workunit.client.0.trial128.stderr:142874: 3' 2026-02-17T11:41:21.325 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:41:21.325 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:41:21.325 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:41:21.325 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836522 2026-02-17T11:41:21.325 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836522 2026-02-17T11:41:21.325 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522' 2026-02-17T11:41:23.994 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpersph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:41:23.995 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=339302416386 2026-02-17T11:41:23.995 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 339302416386 2026-02-17T11:41:23.995 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521 1-339302416386' 2026-02-17T11:41:23.995 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:41:23.995 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:41:23.995 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=283467841547 2026-02-17T11:41:23.995 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 283467841547 2026-02-17T11:41:23.995 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521 1-339302416386 2-283467841547' 2026-02-17T11:41:23.995 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:41:23.996 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:41:23.996 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=356482285570 2026-02-17T11:41:23.996 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 356482285570 2026-02-17T11:41:23.996 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521 1-339302416386 2-283467841547 3-356482285570' 2026-02-17T11:41:23.996 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:23.996 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836521 2026-02-17T11:41:23.996 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:23.997 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:41:23.997 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836521 2026-02-17T11:41:23.997 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:23.997 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836521 2026-02-17T11:41:23.997 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836521' 2026-02-17T11:41:23.997 INFO:tasks.workunit.client.0.trial128.stderr:142877: waiting osd.0 seq 21474836521 2026-02-17T11:41:23.997 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:41:23.998 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836520 -lt 21474836521 2026-02-17T11:41:23.998 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:41:23.998 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:41:23.998 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:41:23.998 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836520 -lt 21474836521 2026-02-17T11:41:23.998 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:41:23.998 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:41:23.998 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:41:23.998 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836522 -lt 21474836521 2026-02-17T11:41:23.998 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:24.017 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: e.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:41:24.018 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:41:24.018 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=339302416387 2026-02-17T11:41:24.018 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 339302416387 2026-02-17T11:41:24.018 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-339302416387' 2026-02-17T11:41:24.018 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:41:24.018 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:41:24.018 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=283467841548 2026-02-17T11:41:24.018 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 283467841548 2026-02-17T11:41:24.018 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-339302416387 2-283467841548' 2026-02-17T11:41:24.019 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:41:24.019 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:41:24.019 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=356482285571 2026-02-17T11:41:24.019 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 356482285571 2026-02-17T11:41:24.019 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-339302416387 2-283467841548 3-356482285571' 2026-02-17T11:41:24.019 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:24.019 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836522 2026-02-17T11:41:24.019 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:24.019 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:41:24.019 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836522 2026-02-17T11:41:24.020 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:24.020 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836522 2026-02-17T11:41:24.020 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836522' 2026-02-17T11:41:24.020 INFO:tasks.workunit.client.0.trial128.stderr:142874: waiting osd.0 seq 21474836522 2026-02-17T11:41:24.020 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:41:24.020 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836520 -lt 21474836522 2026-02-17T11:41:24.020 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:41:24.020 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:41:24.020 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:41:24.020 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836520 -lt 21474836522 2026-02-17T11:41:24.021 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:41:24.021 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:41:24.021 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:41:24.021 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836522 -lt 21474836522 2026-02-17T11:41:24.648 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $cho 1-339302416386 2026-02-17T11:41:24.648 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:24.648 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:41:24.648 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-339302416386 2026-02-17T11:41:24.648 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:24.649 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=339302416386 2026-02-17T11:41:24.649 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 339302416386' 2026-02-17T11:41:24.649 INFO:tasks.workunit.client.0.trial128.stderr:142877: waiting osd.1 seq 339302416386 2026-02-17T11:41:24.649 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:41:24.649 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 339302416387 -lt 339302416386 2026-02-17T11:41:24.649 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:24.649 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-283467841547 2026-02-17T11:41:24.649 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:24.650 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:41:24.650 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-283467841547 2026-02-17T11:41:24.650 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:24.650 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=283467841547 2026-02-17T11:41:24.650 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 283467841547' 2026-02-17T11:41:24.650 INFO:tasks.workunit.client.0.trial128.stderr:142877: waiting osd.2 seq 283467841547 2026-02-17T11:41:24.650 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:41:24.651 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 283467841548 -lt 283467841547 2026-02-17T11:41:24.651 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:24.651 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-356482285570 2026-02-17T11:41:24.651 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:24.651 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:41:24.651 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-356482285570 2026-02-17T11:41:24.651 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:24.652 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=356482285570 2026-02-17T11:41:24.652 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 356482285570' 2026-02-17T11:41:24.652 INFO:tasks.workunit.client.0.trial128.stderr:142877: waiting osd.3 seq 356482285570 2026-02-17T11:41:24.652 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:41:24.652 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 356482285571 -lt 356482285570 2026-02-17T11:41:24.652 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:41:24.652 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:41:24.652 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:41:24.653 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:41:24.653 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:41:24.690 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-seqs 2026-02-17T11:41:24.690 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-339302416387 2026-02-17T11:41:24.690 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:24.690 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:41:24.690 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-339302416387 2026-02-17T11:41:24.690 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:24.690 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=339302416387 2026-02-17T11:41:24.690 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 339302416387' 2026-02-17T11:41:24.691 INFO:tasks.workunit.client.0.trial128.stderr:142874: waiting osd.1 seq 339302416387 2026-02-17T11:41:24.691 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:41:24.691 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 339302416387 -lt 339302416387 2026-02-17T11:41:24.691 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:24.691 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-283467841548 2026-02-17T11:41:24.691 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:24.691 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:41:24.691 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-283467841548 2026-02-17T11:41:24.691 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:24.691 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=283467841548 2026-02-17T11:41:24.691 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 283467841548' 2026-02-17T11:41:24.691 INFO:tasks.workunit.client.0.trial128.stderr:142874: waiting osd.2 seq 283467841548 2026-02-17T11:41:24.692 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:41:24.692 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 283467841548 -lt 283467841548 2026-02-17T11:41:24.692 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:41:24.692 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-356482285571 2026-02-17T11:41:24.692 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:41:24.692 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:41:24.692 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-356482285571 2026-02-17T11:41:24.692 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:41:24.692 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=356482285571 2026-02-17T11:41:24.692 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 356482285571' 2026-02-17T11:41:24.692 INFO:tasks.workunit.client.0.trial128.stderr:142874: waiting osd.3 seq 356482285571 2026-02-17T11:41:24.692 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:41:24.693 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 356482285571 -lt 356482285571 2026-02-17T11:41:24.693 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:41:24.693 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:41:24.693 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:41:24.693 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:41:24.980 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/cephhelpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:41:24.981 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:41:24.981 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:41:24.981 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:41:24.981 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:41:24.981 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:41:24.981 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:41:24.981 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:41:24.981 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:41:24.981 INFO:tasks.workunit.client.0.trial128.stderr:142877: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:41:24.981 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:41:24.981 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:41:24.982 INFO:tasks.workunit.client.0.trial128.stderr:142877: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:41:24.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:41:25.033 INFO:tasks.workunit.client.0.trial128.stderr:-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:41:25.034 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:41:25.034 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:41:25.034 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:41:25.034 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:41:25.034 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:41:25.034 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:41:25.034 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:41:25.034 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:41:25.034 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:41:25.035 INFO:tasks.workunit.client.0.trial128.stderr:142874: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:41:25.035 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:41:25.035 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:41:25.035 INFO:tasks.workunit.client.0.trial128.stderr:142874: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:41:25.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:41:25.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:41:25.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 142873 2026-02-17T11:41:25.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T11:41:25.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T11:41:25.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T11:41:25.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:212: corrupt_and_repair_two: return_code=0 2026-02-17T11:41:25.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:213: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T11:41:25.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:41:25.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:41:25.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T11:41:25.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:41:25.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:41:25.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:41:25.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:41:25.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:41:25.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:41:25.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:41:25.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:41:25.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:41:25.367 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:41:25.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:41:25.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:41:25.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:41:25.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:41:25.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:41:25.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:41:25.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:41:25.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:41:25.373 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:41:25.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:41:25.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:41:25.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:41:25.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:41:25.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:41:25.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:41:25.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:41:25.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:41:25.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:41:25.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T11:41:25.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T11:41:25.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T11:41:25.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T11:41:25.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:41:25.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:41:25.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:41:25.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:41:25.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:41:25.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:41:25.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:41:25.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:41:25.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:41:25.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:41:25.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:41:25.409 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:41:25.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:41:25.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:41:25.411 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:41:25.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:41:25.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:41:25.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:41:25.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:41:25.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:41:25.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:41:25.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:41:25.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:41:25.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:41:25.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:41:25.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:41:25.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:41:25.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:41:25.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T11:41:25.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:41:25.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:41:25.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:41:25.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T11:41:25.422 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:41:25.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T11:41:25.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:41:25.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T11:41:25.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_and_repair_jerasure_overwrites td/osd-scrub-repair 2026-02-17T11:41:25.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:819: TEST_corrupt_and_repair_jerasure_overwrites: '[' true = true ']' 2026-02-17T11:41:25.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:820: TEST_corrupt_and_repair_jerasure_overwrites: corrupt_and_repair_jerasure td/osd-scrub-repair true 2026-02-17T11:41:25.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:798: corrupt_and_repair_jerasure: local dir=td/osd-scrub-repair 2026-02-17T11:41:25.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:799: corrupt_and_repair_jerasure: local allow_overwrites=true 2026-02-17T11:41:25.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:800: corrupt_and_repair_jerasure: local poolname=ecpool 2026-02-17T11:41:25.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:802: corrupt_and_repair_jerasure: run_mon td/osd-scrub-repair a 2026-02-17T11:41:25.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T11:41:25.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T11:41:25.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T11:41:25.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T11:41:25.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T11:41:25.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T11:41:25.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T11:41:25.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:41:25.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:41:25.446 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:41:25.446 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:41:25.447 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:41:25.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:25.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T11:41:25.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T11:41:25.474 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T11:41:25.474 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:41:25.474 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:41:25.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T11:41:25.476 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T11:41:25.476 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:41:25.476 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:41:25.476 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:41:25.476 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:41:25.477 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:41:25.477 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:41:25.477 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:41:25.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:41:25.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T11:41:25.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T11:41:25.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:41:25.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:41:25.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T11:41:25.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T11:41:25.531 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:41:25.531 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:41:25.531 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:41:25.531 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:41:25.531 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:41:25.531 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:41:25.531 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:41:25.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:41:25.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T11:41:25.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:803: corrupt_and_repair_jerasure: run_mgr td/osd-scrub-repair x 2026-02-17T11:41:25.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T11:41:25.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T11:41:25.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T11:41:25.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T11:41:25.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T11:41:25.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T11:41:25.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T11:41:25.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:41:25.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:41:25.677 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:41:25.677 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:41:25.677 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:41:25.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:25.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:41:25.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:41:25.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: seq 0 3 2026-02-17T11:41:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-17T11:41:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 0 2026-02-17T11:41:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:41:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:41:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T11:41:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:41:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:41:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:41:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:41:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:41:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:41:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:41:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:41:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:41:25.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:41:25.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:41:25.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:41:25.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:41:25.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:41:25.701 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:41:25.701 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:41:25.701 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:41:25.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:25.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:25.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:41:25.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:41:25.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:41:25.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:41:25.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:41:25.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:41:25.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:41:25.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:41:25.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:41:25.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:41:25.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:41:25.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:41:25.703 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:41:25.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=84cbeca5-6def-4db7-a06a-cfe4e3d3b353 2026-02-17T11:41:25.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 84cbeca5-6def-4db7-a06a-cfe4e3d3b353' 2026-02-17T11:41:25.704 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 84cbeca5-6def-4db7-a06a-cfe4e3d3b353 2026-02-17T11:41:25.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:41:25.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDlU5Rpz2dAKhAAq4PrMPRTGTabPb4aDih59A== 2026-02-17T11:41:25.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDlU5Rpz2dAKhAAq4PrMPRTGTabPb4aDih59A=="}' 2026-02-17T11:41:25.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 84cbeca5-6def-4db7-a06a-cfe4e3d3b353 -i td/osd-scrub-repair/0/new.json 2026-02-17T11:41:25.807 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:41:25.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T11:41:25.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDlU5Rpz2dAKhAAq4PrMPRTGTabPb4aDih59A== --osd-uuid 84cbeca5-6def-4db7-a06a-cfe4e3d3b353 2026-02-17T11:41:25.830 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:25.820+0000 7f680045c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:25.832 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:25.824+0000 7f680045c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:25.833 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:25.824+0000 7f680045c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:25.833 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:25.824+0000 7f680045c8c0 -1 bdev(0x55bf69c73c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:41:25.833 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:25.824+0000 7f680045c8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T11:41:28.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T11:41:28.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:41:28.629 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:41:28.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T11:41:28.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:41:28.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T11:41:28.818 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:41:28.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:41:28.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:41:28.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:41:28.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:41:28.841 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:28.832+0000 7f09c90d08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:28.842 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:28.832+0000 7f09c90d08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:28.843 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:28.836+0000 7f09c90d08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:28.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T11:41:28.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:41:28.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:41:28.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:41:28.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:41:28.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:28.974 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:41:28.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:41:28.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:28.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:41:29.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:30.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:30.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:30.125 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:41:30.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:41:30.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:30.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:41:30.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:30.316 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:30.308+0000 7f09c90d08c0 -1 Falling back to public interface 2026-02-17T11:41:31.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:31.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:31.282 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:41:31.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:41:31.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:31.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:41:31.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:32.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:32.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:32.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:41:32.433 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:41:32.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:32.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:41:32.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:33.300 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:33.292+0000 7f09c90d08c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:41:33.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:33.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:33.586 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:41:33.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:41:33.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:33.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:41:33.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:34.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:34.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:34.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:41:34.742 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:41:34.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:34.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:41:34.892 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3145363187,v1:127.0.0.1:6803/3145363187] [v2:127.0.0.1:6804/3145363187,v1:127.0.0.1:6805/3145363187] exists,up 84cbeca5-6def-4db7-a06a-cfe4e3d3b353 2026-02-17T11:41:34.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:41:34.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:41:34.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:41:34.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-17T11:41:34.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 1 2026-02-17T11:41:34.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:41:34.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:41:34.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T11:41:34.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:41:34.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:41:34.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:41:34.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:41:34.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:41:34.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:41:34.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:41:34.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:41:34.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:41:34.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:41:34.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:41:34.897 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:41:34.897 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:41:34.897 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:41:34.898 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:41:34.898 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:41:34.898 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:41:34.898 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:34.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:34.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:41:34.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:41:34.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:41:34.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:41:34.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:41:34.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:41:34.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:41:34.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:41:34.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:41:34.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:41:34.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:41:34.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:41:34.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:41:34.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=0107cbfc-eed6-42f4-b707-bfb3f3155b29 2026-02-17T11:41:34.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 0107cbfc-eed6-42f4-b707-bfb3f3155b29' 2026-02-17T11:41:34.901 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 0107cbfc-eed6-42f4-b707-bfb3f3155b29 2026-02-17T11:41:34.902 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:41:34.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDuU5RpOobQNRAAVcwVwMKiy/50+nwUVCvm8w== 2026-02-17T11:41:34.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDuU5RpOobQNRAAVcwVwMKiy/50+nwUVCvm8w=="}' 2026-02-17T11:41:34.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 0107cbfc-eed6-42f4-b707-bfb3f3155b29 -i td/osd-scrub-repair/1/new.json 2026-02-17T11:41:35.060 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:41:35.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T11:41:35.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDuU5RpOobQNRAAVcwVwMKiy/50+nwUVCvm8w== --osd-uuid 0107cbfc-eed6-42f4-b707-bfb3f3155b29 2026-02-17T11:41:35.086 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:35.076+0000 7f7cedbc78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:35.088 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:35.080+0000 7f7cedbc78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:35.089 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:35.080+0000 7f7cedbc78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:35.089 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:35.080+0000 7f7cedbc78c0 -1 bdev(0x55bbc2ffdc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:41:35.089 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:35.080+0000 7f7cedbc78c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T11:41:37.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T11:41:37.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:41:37.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T11:41:37.381 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:41:37.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:41:37.570 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:41:37.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T11:41:37.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:41:37.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:41:37.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:41:37.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:41:37.592 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:37.584+0000 7f117ed5d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:37.594 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:37.584+0000 7f117ed5d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:37.595 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:37.588+0000 7f117ed5d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:37.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T11:41:37.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:41:37.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:41:37.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:41:37.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:41:37.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:37.725 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:41:37.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:41:37.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:37.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:41:37.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:38.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:38.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:38.882 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:41:38.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:41:38.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:38.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:41:39.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:39.297 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:39.288+0000 7f117ed5d8c0 -1 Falling back to public interface 2026-02-17T11:41:40.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:40.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:40.033 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:41:40.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:41:40.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:40.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:41:40.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:41.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:41.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:41.192 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:41:41.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:41:41.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:41.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:41:41.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:42.266 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:42.255+0000 7f117ed5d8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:41:42.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:42.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:42.341 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:41:42.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:41:42.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:42.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:41:42.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:43.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:43.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:43.494 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:41:43.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:41:43.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:43.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:41:43.646 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/149315158,v1:127.0.0.1:6811/149315158] [v2:127.0.0.1:6812/149315158,v1:127.0.0.1:6813/149315158] exists,up 0107cbfc-eed6-42f4-b707-bfb3f3155b29 2026-02-17T11:41:43.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:41:43.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:41:43.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:41:43.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-17T11:41:43.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 2 2026-02-17T11:41:43.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:41:43.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:41:43.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T11:41:43.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:41:43.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:41:43.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:41:43.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:41:43.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:41:43.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:41:43.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:41:43.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:41:43.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:41:43.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:41:43.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:41:43.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:41:43.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:41:43.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:41:43.651 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:41:43.651 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:41:43.651 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:41:43.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:43.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:43.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:41:43.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:41:43.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:41:43.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:41:43.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:41:43.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:41:43.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:41:43.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:41:43.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:41:43.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:41:43.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:41:43.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:41:43.653 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:41:43.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=9deb3e7d-4fba-421d-9345-6e38593da42b 2026-02-17T11:41:43.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 9deb3e7d-4fba-421d-9345-6e38593da42b' 2026-02-17T11:41:43.654 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 9deb3e7d-4fba-421d-9345-6e38593da42b 2026-02-17T11:41:43.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:41:43.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQD3U5RpZdsdJxAA0FLYG7pM8u+ccz0SRbuh4A== 2026-02-17T11:41:43.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQD3U5RpZdsdJxAA0FLYG7pM8u+ccz0SRbuh4A=="}' 2026-02-17T11:41:43.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 9deb3e7d-4fba-421d-9345-6e38593da42b -i td/osd-scrub-repair/2/new.json 2026-02-17T11:41:43.818 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:41:43.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T11:41:43.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD3U5RpZdsdJxAA0FLYG7pM8u+ccz0SRbuh4A== --osd-uuid 9deb3e7d-4fba-421d-9345-6e38593da42b 2026-02-17T11:41:43.844 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:43.835+0000 7f7c8efaf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:43.845 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:43.835+0000 7f7c8efaf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:43.846 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:43.835+0000 7f7c8efaf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:43.846 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:43.835+0000 7f7c8efaf8c0 -1 bdev(0x55d3f6a2dc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T11:41:43.846 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:43.835+0000 7f7c8efaf8c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T11:41:46.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T11:41:46.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:41:46.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T11:41:46.153 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T11:41:46.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:41:46.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T11:41:46.342 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T11:41:46.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:41:46.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:41:46.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:41:46.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:41:46.364 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:46.355+0000 7ff9ae3fd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:46.365 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:46.355+0000 7ff9ae3fd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:46.366 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:46.355+0000 7ff9ae3fd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:46.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T11:41:46.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:41:46.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:41:46.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:41:46.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:41:46.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:46.499 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:41:46.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:41:46.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:46.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:41:46.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:47.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:47.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:47.647 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:41:47.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:41:47.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:47.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:41:47.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:47.853 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:47.843+0000 7ff9ae3fd8c0 -1 Falling back to public interface 2026-02-17T11:41:48.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:48.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:48.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:41:48.799 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:41:48.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:48.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:41:48.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:49.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:49.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:49.951 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:41:49.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:41:49.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:49.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:41:50.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:51.079 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:51.067+0000 7ff9ae3fd8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T11:41:51.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:51.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:51.102 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:41:51.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:41:51.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:51.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:41:51.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:52.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:52.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:52.260 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:41:52.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:41:52.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:52.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:41:52.410 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1274356168,v1:127.0.0.1:6819/1274356168] [v2:127.0.0.1:6820/1274356168,v1:127.0.0.1:6821/1274356168] exists,up 9deb3e7d-4fba-421d-9345-6e38593da42b 2026-02-17T11:41:52.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:41:52.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:41:52.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:41:52.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:804: corrupt_and_repair_jerasure: for id in $(seq 0 3) 2026-02-17T11:41:52.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:805: corrupt_and_repair_jerasure: run_osd td/osd-scrub-repair 3 2026-02-17T11:41:52.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:41:52.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:41:52.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T11:41:52.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:41:52.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:41:52.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:41:52.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:41:52.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:41:52.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:41:52.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:41:52.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:41:52.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:41:52.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:41:52.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:41:52.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:41:52.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:41:52.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:41:52.413 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:41:52.413 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:41:52.413 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:41:52.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:52.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:41:52.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:41:52.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:41:52.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:41:52.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:41:52.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:41:52.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:41:52.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:41:52.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:41:52.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:41:52.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:41:52.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:41:52.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:41:52.415 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:41:52.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=bdb03fba-ae78-43d3-8520-91782827db02 2026-02-17T11:41:52.415 INFO:tasks.workunit.client.0.trial128.stdout:add osd3 bdb03fba-ae78-43d3-8520-91782827db02 2026-02-17T11:41:52.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 bdb03fba-ae78-43d3-8520-91782827db02' 2026-02-17T11:41:52.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:41:52.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAAVJRppX8HGRAAl5kdyjZGBFCd0O8iyOe5eA== 2026-02-17T11:41:52.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAAVJRppX8HGRAAl5kdyjZGBFCd0O8iyOe5eA=="}' 2026-02-17T11:41:52.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new bdb03fba-ae78-43d3-8520-91782827db02 -i td/osd-scrub-repair/3/new.json 2026-02-17T11:41:52.575 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:41:52.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-17T11:41:52.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAAVJRppX8HGRAAl5kdyjZGBFCd0O8iyOe5eA== --osd-uuid bdb03fba-ae78-43d3-8520-91782827db02 2026-02-17T11:41:52.601 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:52.591+0000 7f50eecef8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:52.602 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:52.591+0000 7f50eecef8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:52.603 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:52.595+0000 7f50eecef8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:52.604 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:52.595+0000 7f50eecef8c0 -1 bdev(0x55af46513c00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-17T11:41:52.604 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:52.595+0000 7f50eecef8c0 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-17T11:41:54.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-17T11:41:54.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:41:54.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T11:41:54.881 INFO:tasks.workunit.client.0.trial128.stdout:adding osd3 key to auth repository 2026-02-17T11:41:54.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:41:55.064 INFO:tasks.workunit.client.0.trial128.stdout:start osd.3 2026-02-17T11:41:55.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T11:41:55.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:41:55.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:41:55.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:41:55.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:41:55.080 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:55.071+0000 7fe2a8ac18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:55.082 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:55.071+0000 7fe2a8ac18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:55.083 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:55.071+0000 7fe2a8ac18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:41:55.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T11:41:55.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:41:55.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:41:55.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:41:55.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:41:55.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:55.213 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:41:55.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:41:55.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:55.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:41:55.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:56.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:56.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:56.369 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:41:56.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:41:56.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:56.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:41:56.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:56.589 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:56.579+0000 7fe2a8ac18c0 -1 Falling back to public interface 2026-02-17T11:41:57.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:57.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:57.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:41:57.526 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:41:57.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:57.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:41:57.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:58.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:58.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:58.681 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:41:58.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:41:58.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:58.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:41:58.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:41:59.563 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:41:59.551+0000 7fe2a8ac18c0 -1 osd.3 0 log_to_monitors true 2026-02-17T11:41:59.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:41:59.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:41:59.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:41:59.837 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:41:59.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:41:59.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:41:59.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:00.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:00.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:00.992 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:42:00.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:42:00.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:00.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:42:01.147 INFO:tasks.workunit.client.0.trial128.stdout:osd.3 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3543797145,v1:127.0.0.1:6827/3543797145] [v2:127.0.0.1:6828/3543797145,v1:127.0.0.1:6829/3543797145] exists,up bdb03fba-ae78-43d3-8520-91782827db02 2026-02-17T11:42:01.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:42:01.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:42:01.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:42:01.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:807: corrupt_and_repair_jerasure: create_rbd_pool 2026-02-17T11:42:01.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T11:42:01.289 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' does not exist 2026-02-17T11:42:01.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T11:42:01.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T11:42:01.512 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T11:42:01.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:42:02.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T11:42:02.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:808: corrupt_and_repair_jerasure: wait_for_clean 2026-02-17T11:42:02.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:42:02.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:42:02.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:42:02.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:42:02.846 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:42:02.847 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:42:02.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:42:02.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:42:02.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:42:02.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:42:02.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:42:02.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:42:02.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:42:02.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:42:02.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:42:03.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:42:03.109 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:42:03.109 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:42:03.109 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T11:42:03.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:42:03.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:03.110 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:42:03.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T11:42:03.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T11:42:03.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T11:42:03.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:03.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:42:03.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-17T11:42:03.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-17T11:42:03.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669' 2026-02-17T11:42:03.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:03.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:42:03.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574852 2026-02-17T11:42:03.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574852 2026-02-17T11:42:03.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-55834574852' 2026-02-17T11:42:03.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:03.329 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:42:03.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=73014444034 2026-02-17T11:42:03.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 73014444034 2026-02-17T11:42:03.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-55834574852 3-73014444034' 2026-02-17T11:42:03.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:03.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T11:42:03.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:03.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:42:03.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T11:42:03.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:03.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T11:42:03.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T11:42:03.405 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836487 2026-02-17T11:42:03.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:42:03.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-17T11:42:03.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:42:04.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:42:04.557 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:42:04.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T11:42:04.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:04.714 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-17T11:42:04.715 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:04.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:42:04.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-17T11:42:04.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:04.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-17T11:42:04.718 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705669 2026-02-17T11:42:04.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-17T11:42:04.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:42:04.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705670 -lt 38654705669 2026-02-17T11:42:04.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:04.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574852 2026-02-17T11:42:04.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:04.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:42:04.871 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574852 2026-02-17T11:42:04.871 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:04.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574852 2026-02-17T11:42:04.873 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574852 2026-02-17T11:42:04.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574852' 2026-02-17T11:42:04.873 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:42:05.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-17T11:42:05.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:05.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-73014444034 2026-02-17T11:42:05.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:05.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:42:05.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-73014444034 2026-02-17T11:42:05.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:05.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=73014444034 2026-02-17T11:42:05.034 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.3 seq 73014444034 2026-02-17T11:42:05.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 73014444034' 2026-02-17T11:42:05.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:42:05.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 73014444034 -lt 73014444034 2026-02-17T11:42:05.184 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:42:05.184 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:42:05.184 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:42:05.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T11:42:05.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:42:05.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:42:05.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:42:05.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:42:05.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:42:05.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:42:05.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:42:05.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T11:42:05.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:42:05.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:42:05.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:42:05.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T11:42:05.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:42:05.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:42:05.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:810: corrupt_and_repair_jerasure: create_ec_pool ecpool true k=2 m=2 2026-02-17T11:42:05.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T11:42:05.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T11:42:05.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=true 2026-02-17T11:42:05.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T11:42:05.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=2 2026-02-17T11:42:05.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T11:42:05.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T11:42:06.200 INFO:tasks.workunit.client.0.trial128.stderr:pool 'ecpool' created 2026-02-17T11:42:06.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:42:07.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' true = true ']' 2026-02-17T11:42:07.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2508: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-17T11:42:07.393 INFO:tasks.workunit.client.0.trial128.stderr:set pool 2 allow_ec_overwrites to true 2026-02-17T11:42:07.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T11:42:07.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:42:07.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:42:07.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:42:07.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:42:07.410 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:42:07.410 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:42:07.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:42:07.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:42:07.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:42:07.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:42:07.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:42:07.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:42:07.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:42:07.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:42:07.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:42:07.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:42:07.664 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:42:07.664 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:42:07.664 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T11:42:07.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:42:07.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:07.664 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:42:07.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-17T11:42:07.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-17T11:42:07.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-17T11:42:07.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:07.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:42:07.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705671 2026-02-17T11:42:07.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705671 2026-02-17T11:42:07.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705671' 2026-02-17T11:42:07.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:07.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:42:07.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574854 2026-02-17T11:42:07.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574854 2026-02-17T11:42:07.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705671 2-55834574854' 2026-02-17T11:42:07.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:07.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:42:07.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=73014444036 2026-02-17T11:42:07.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 73014444036 2026-02-17T11:42:07.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705671 2-55834574854 3-73014444036' 2026-02-17T11:42:07.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:07.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-17T11:42:07.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:07.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:42:07.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-17T11:42:07.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:07.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-17T11:42:07.957 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836489 2026-02-17T11:42:07.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-17T11:42:07.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:42:08.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-17T11:42:08.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:08.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705671 2026-02-17T11:42:08.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:08.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:42:08.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705671 2026-02-17T11:42:08.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:08.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705671 2026-02-17T11:42:08.125 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705671 2026-02-17T11:42:08.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-17T11:42:08.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:42:08.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705671 -lt 38654705671 2026-02-17T11:42:08.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:08.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574854 2026-02-17T11:42:08.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:08.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:42:08.279 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574854 2026-02-17T11:42:08.279 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:08.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574854 2026-02-17T11:42:08.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574854' 2026-02-17T11:42:08.281 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574854 2026-02-17T11:42:08.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:42:08.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-17T11:42:08.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:08.434 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-73014444036 2026-02-17T11:42:08.434 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:08.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:42:08.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-73014444036 2026-02-17T11:42:08.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:08.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=73014444036 2026-02-17T11:42:08.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 73014444036' 2026-02-17T11:42:08.437 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.3 seq 73014444036 2026-02-17T11:42:08.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:42:08.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 73014444036 -lt 73014444036 2026-02-17T11:42:08.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:42:08.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:42:08.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:42:08.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:42:08.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:42:08.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:42:08.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:42:08.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:42:08.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:42:08.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:42:08.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:42:08.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:42:08.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:42:08.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:42:08.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:42:09.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:42:09.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:42:09.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:42:09.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T11:42:09.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:811: corrupt_and_repair_jerasure: corrupt_and_repair_erasure_coded td/osd-scrub-repair ecpool 2026-02-17T11:42:09.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:248: corrupt_and_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T11:42:09.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:249: corrupt_and_repair_erasure_coded: local poolname=ecpool 2026-02-17T11:42:09.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:251: corrupt_and_repair_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-17T11:42:09.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:42:09.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:42:09.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T11:42:09.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:42:09.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:42:09.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:42:09.300 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:42:09.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:42:09.504 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:42:09.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:42:09.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:42:09.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T11:42:09.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_erasure_coded: get_primary ecpool SOMETHING 2026-02-17T11:42:09.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T11:42:09.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:42:09.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:42:09.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:42:09.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_erasure_coded: local primary=3 2026-02-17T11:42:09.694 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: get_osds ecpool SOMETHING 2026-02-17T11:42:09.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: sed -e s/3// 2026-02-17T11:42:09.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T11:42:09.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T11:42:09.695 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:42:09.695 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T11:42:09.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T11:42:09.844 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:42:09.844 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:42:09.844 INFO:tasks.workunit.client.0.trial128.stderr:0' 2026-02-17T11:42:09.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 1 2 0 2026-02-17T11:42:09.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: osds=('1' '2' '0') 2026-02-17T11:42:09.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: local -a osds 2026-02-17T11:42:09.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:255: corrupt_and_repair_erasure_coded: local not_primary_first=1 2026-02-17T11:42:09.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:256: corrupt_and_repair_erasure_coded: local not_primary_second=2 2026-02-17T11:42:09.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:259: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 3 2026-02-17T11:42:09.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T11:42:09.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=ecpool 2026-02-17T11:42:09.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=3 2026-02-17T11:42:09.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:42:09.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:42:09.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:42:09.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T11:42:09.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:42:09.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:42:09.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:42:09.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:42:09.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T11:42:09.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:42:09.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T11:42:09.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:42:09.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:42:09.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:42:09.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:42:09.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:42:11.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:42:11.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:42:11.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:42:11.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:42:11.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T11:42:11.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:42:11.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:42:11.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T11:42:11.805 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#2:eb822e21:::SOMETHING:head# 2026-02-17T11:42:12.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T11:42:12.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:42:12.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:42:12.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T11:42:12.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:42:12.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:42:12.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:42:12.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:42:12.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:42:12.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:42:12.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:42:12.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:42:12.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:42:12.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:42:12.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:42:12.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:42:12.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:42:12.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:42:12.344 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:42:12.344 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:42:12.344 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:42:12.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:42:12.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:42:12.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:42:12.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:42:12.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:42:12.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:42:12.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:42:12.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:42:12.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:42:12.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:42:12.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:42:12.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:42:12.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T11:42:12.347 INFO:tasks.workunit.client.0.trial128.stderr:start osd.3 2026-02-17T11:42:12.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:42:12.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T11:42:12.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T11:42:12.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:42:12.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:42:12.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:42:12.368 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:12.355+0000 7f05a6cb18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:42:12.369 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:12.359+0000 7f05a6cb18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:42:12.370 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:12.359+0000 7f05a6cb18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:42:12.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T11:42:12.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:42:12.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:42:12.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:42:12.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:42:12.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:12.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:42:12.502 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:42:12.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:12.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:42:12.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:13.609 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:13.599+0000 7f05a6cb18c0 -1 Falling back to public interface 2026-02-17T11:42:13.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:13.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:13.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:42:13.657 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:42:13.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:13.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:42:13.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:14.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:14.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:14.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:42:14.808 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:42:14.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:14.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:42:14.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:15.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:15.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:15.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:42:15.961 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:42:15.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:15.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:42:16.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:16.631 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:16.619+0000 7f05a6cb18c0 -1 osd.3 33 log_to_monitors true 2026-02-17T11:42:17.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:17.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:17.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:42:17.114 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:42:17.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:17.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:42:17.260 INFO:tasks.workunit.client.0.trial128.stderr:osd.3 up in weight 1 up_from 37 up_thru 37 down_at 34 last_clean_interval [17,33) [v2:127.0.0.1:6826/1316513451,v1:127.0.0.1:6827/1316513451] [v2:127.0.0.1:6828/1316513451,v1:127.0.0.1:6829/1316513451] exists,up bdb03fba-ae78-43d3-8520-91782827db02 2026-02-17T11:42:17.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:42:17.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:42:17.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:42:17.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:42:17.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:42:17.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:42:17.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:42:17.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:42:17.263 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:42:17.263 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:42:17.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:42:17.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:42:17.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:42:17.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:42:17.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:42:17.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:42:17.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:42:17.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:42:17.373 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:42:17.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:42:17.528 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:42:17.528 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:42:17.528 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T11:42:17.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:42:17.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:17.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:42:17.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836492 2026-02-17T11:42:17.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836492 2026-02-17T11:42:17.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492' 2026-02-17T11:42:17.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:17.599 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:42:17.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705674 2026-02-17T11:42:17.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705674 2026-02-17T11:42:17.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-38654705674' 2026-02-17T11:42:17.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:17.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:42:17.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574857 2026-02-17T11:42:17.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574857 2026-02-17T11:42:17.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-38654705674 2-55834574857' 2026-02-17T11:42:17.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:17.740 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:42:17.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789954 2026-02-17T11:42:17.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789954 2026-02-17T11:42:17.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-38654705674 2-55834574857 3-158913789954' 2026-02-17T11:42:17.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:17.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836492 2026-02-17T11:42:17.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:17.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:42:17.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836492 2026-02-17T11:42:17.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:17.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836492 2026-02-17T11:42:17.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-17T11:42:17.818 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836492 2026-02-17T11:42:17.818 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:42:17.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836492 2026-02-17T11:42:17.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:42:18.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:42:18.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:42:19.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836492 2026-02-17T11:42:19.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:19.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705674 2026-02-17T11:42:19.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:19.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:42:19.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705674 2026-02-17T11:42:19.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:19.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705674 2026-02-17T11:42:19.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-17T11:42:19.139 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705674 2026-02-17T11:42:19.139 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:42:19.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705674 -lt 38654705674 2026-02-17T11:42:19.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:19.303 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574857 2026-02-17T11:42:19.303 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:19.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:42:19.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574857 2026-02-17T11:42:19.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:19.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574857 2026-02-17T11:42:19.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574857' 2026-02-17T11:42:19.306 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 55834574857 2026-02-17T11:42:19.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:42:19.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-17T11:42:19.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:19.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-158913789954 2026-02-17T11:42:19.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:19.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:42:19.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-158913789954 2026-02-17T11:42:19.476 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:19.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789954 2026-02-17T11:42:19.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 158913789954' 2026-02-17T11:42:19.477 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.3 seq 158913789954 2026-02-17T11:42:19.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:42:19.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789954 -lt 158913789954 2026-02-17T11:42:19.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:42:19.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:42:19.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:42:19.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:42:19.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:42:19.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:42:19.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:42:19.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:42:19.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:42:19.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:42:19.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:42:19.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:42:19.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:42:19.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:42:19.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:42:20.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:42:20.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:42:20.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:42:20.149 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-17T11:42:20.149 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:42:20.149 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:42:20.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:42:20.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:42:20.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=2.0 2026-02-17T11:42:20.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 2.0 2026-02-17T11:42:20.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T11:42:20.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T11:42:20.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:42:20.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:42:20.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:42:20.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:42:20.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:42:06.193998+0000 2026-02-17T11:42:20.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T11:42:20.601 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T11:42:20.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T11:42:06.193998+0000 2026-02-17T11:42:20.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:42:20.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:42:06.193998+0000 2026-02-17T11:42:20.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:42:20.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:42:20.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:42:20.611 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:42:20.612 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:42:20.612 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:42:20.612 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:42:20.612 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:42:20.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:42:06.193998+0000 '>' 2026-02-17T11:42:06.193998+0000 2026-02-17T11:42:20.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:42:21.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:42:21.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:42:21.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:42:21.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:42:21.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:42:21.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:42:21.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:42:21.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:42:06.193998+0000 '>' 2026-02-17T11:42:06.193998+0000 2026-02-17T11:42:21.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:42:22.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:42:22.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:42:22.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:42:22.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:42:22.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:42:22.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:42:22.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:42:23.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:42:20.631527+0000 '>' 2026-02-17T11:42:06.193998+0000 2026-02-17T11:42:23.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:42:23.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:42:23.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:42:23.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:42:23.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T11:42:23.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:42:23.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:42:23.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:42:23.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:42:23.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T11:42:23.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:42:23.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T11:42:23.078 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:42:23.078 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:42:23.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:42:23.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:42:23.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:42:24.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:42:24.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:42:24.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:42:24.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:42:24.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T11:42:24.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:42:24.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:42:24.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-17T11:42:24.702 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-17T11:42:24.703 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:42:24.703 INFO:tasks.workunit.client.0.trial128.stdout:hinfo_key 2026-02-17T11:42:24.703 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:42:25.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T11:42:25.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:42:25.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:42:25.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T11:42:25.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:42:25.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:42:25.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:42:25.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:42:25.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:42:25.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:42:25.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:42:25.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:42:25.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:42:25.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:42:25.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:42:25.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:42:25.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:42:25.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:42:25.239 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:42:25.239 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:42:25.239 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:42:25.241 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:42:25.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:42:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:42:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:42:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:42:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:42:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:42:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:42:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:42:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:42:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:42:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:42:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T11:42:25.242 INFO:tasks.workunit.client.0.trial128.stderr:start osd.3 2026-02-17T11:42:25.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:42:25.243 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T11:42:25.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T11:42:25.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:42:25.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:42:25.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:42:25.259 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:25.246+0000 7f20336ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:42:25.260 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:25.250+0000 7f20336ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:42:25.261 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:25.250+0000 7f20336ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:42:25.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T11:42:25.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:42:25.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:42:25.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:42:25.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:42:25.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:25.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:42:25.394 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:42:25.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:25.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:42:25.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:26.497 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:26.486+0000 7f20336ba8c0 -1 Falling back to public interface 2026-02-17T11:42:26.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:26.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:26.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:42:26.550 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:42:26.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:26.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:42:26.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:27.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:27.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:27.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:42:27.701 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:42:27.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:27.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:42:27.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:28.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:28.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:28.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:42:28.856 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:42:28.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:28.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:42:29.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:29.496 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:29.486+0000 7f20336ba8c0 -1 osd.3 40 log_to_monitors true 2026-02-17T11:42:30.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:30.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:30.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:42:30.015 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:42:30.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:30.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:42:30.168 INFO:tasks.workunit.client.0.trial128.stderr:osd.3 up in weight 1 up_from 44 up_thru 44 down_at 41 last_clean_interval [37,40) [v2:127.0.0.1:6826/2850561247,v1:127.0.0.1:6827/2850561247] [v2:127.0.0.1:6828/2850561247,v1:127.0.0.1:6829/2850561247] exists,up bdb03fba-ae78-43d3-8520-91782827db02 2026-02-17T11:42:30.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:42:30.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:42:30.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:42:30.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:42:30.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:42:30.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:42:30.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:42:30.169 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:42:30.170 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:42:30.170 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:42:30.170 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:42:30.170 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:42:30.170 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:42:30.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:42:30.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:42:30.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:42:30.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:42:30.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:42:30.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:42:30.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:42:30.435 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:42:30.435 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:42:30.435 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T11:42:30.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:42:30.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:30.435 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:42:30.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836496 2026-02-17T11:42:30.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836496 2026-02-17T11:42:30.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836496' 2026-02-17T11:42:30.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:30.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:42:30.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705678 2026-02-17T11:42:30.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705678 2026-02-17T11:42:30.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836496 1-38654705678' 2026-02-17T11:42:30.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:30.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:42:30.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574860 2026-02-17T11:42:30.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574860 2026-02-17T11:42:30.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836496 1-38654705678 2-55834574860' 2026-02-17T11:42:30.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:30.646 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:42:30.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561026 2026-02-17T11:42:30.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561026 2026-02-17T11:42:30.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836496 1-38654705678 2-55834574860 3-188978561026' 2026-02-17T11:42:30.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:30.722 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836496 2026-02-17T11:42:30.722 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:30.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:42:30.723 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836496 2026-02-17T11:42:30.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:30.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836496 2026-02-17T11:42:30.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836496' 2026-02-17T11:42:30.725 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836496 2026-02-17T11:42:30.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:42:30.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836495 -lt 21474836496 2026-02-17T11:42:30.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:42:31.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:42:31.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:42:32.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836495 -lt 21474836496 2026-02-17T11:42:32.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:42:33.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:42:33.044 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:42:33.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836496 2026-02-17T11:42:33.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:33.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705678 2026-02-17T11:42:33.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:33.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:42:33.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705678 2026-02-17T11:42:33.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:33.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705678 2026-02-17T11:42:33.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705678' 2026-02-17T11:42:33.204 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705678 2026-02-17T11:42:33.204 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:42:33.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705678 -lt 38654705678 2026-02-17T11:42:33.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:33.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574860 2026-02-17T11:42:33.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:33.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:42:33.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574860 2026-02-17T11:42:33.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:33.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574860 2026-02-17T11:42:33.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574860' 2026-02-17T11:42:33.360 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 55834574860 2026-02-17T11:42:33.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:42:33.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574860 -lt 55834574860 2026-02-17T11:42:33.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:33.509 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-188978561026 2026-02-17T11:42:33.509 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:33.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:42:33.511 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-188978561026 2026-02-17T11:42:33.511 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:33.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561026 2026-02-17T11:42:33.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 188978561026' 2026-02-17T11:42:33.513 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.3 seq 188978561026 2026-02-17T11:42:33.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:42:33.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561026 -lt 188978561026 2026-02-17T11:42:33.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:42:33.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:42:33.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:42:33.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:42:33.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:42:33.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:42:33.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:42:33.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:42:33.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:42:33.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:42:33.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:42:34.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:42:34.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:42:34.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:42:34.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:42:34.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:42:34.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:42:34.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:42:34.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:42:34.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:42:34.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:261: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 1 2026-02-17T11:42:34.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T11:42:34.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=ecpool 2026-02-17T11:42:34.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=1 2026-02-17T11:42:34.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:42:34.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:42:34.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:42:34.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:42:34.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:42:34.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:42:34.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:42:34.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:42:34.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:42:34.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:42:34.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:42:34.220 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:42:34.220 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:42:34.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:42:34.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:42:34.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:42:35.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:42:35.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:42:35.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:42:35.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:42:35.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:42:35.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:42:35.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:42:35.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T11:42:36.189 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T11:42:36.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:42:36.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:42:36.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:42:36.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:42:36.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:42:36.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:42:36.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:42:36.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:42:36.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:42:36.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:42:36.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:42:36.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:42:36.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:42:36.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:42:36.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:42:36.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:42:36.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:42:36.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:42:36.736 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:42:36.736 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:42:36.737 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:42:36.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:42:36.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:42:36.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:42:36.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:42:36.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:42:36.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:42:36.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:42:36.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:42:36.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:42:36.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:42:36.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:42:36.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:42:36.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:42:36.739 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:42:36.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:42:36.740 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:42:36.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:42:36.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:42:36.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:42:36.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:42:36.760 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:36.750+0000 7fa106d6e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:42:36.761 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:36.750+0000 7fa106d6e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:42:36.762 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:36.750+0000 7fa106d6e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:42:36.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:42:36.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:42:36.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:42:36.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:42:36.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:42:36.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:36.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:42:36.896 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:42:36.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:36.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:42:37.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:38.006 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:37.994+0000 7fa106d6e8c0 -1 Falling back to public interface 2026-02-17T11:42:38.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:38.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:38.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:42:38.050 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:42:38.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:38.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:42:38.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:39.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:39.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:39.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:42:39.203 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:42:39.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:39.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:42:39.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:40.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:40.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:40.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:42:40.357 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:42:40.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:40.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:42:40.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:41.012 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:41.002+0000 7fa106d6e8c0 -1 osd.1 46 log_to_monitors true 2026-02-17T11:42:41.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:41.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:41.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:42:41.515 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:42:41.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:41.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:42:41.668 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 50 up_thru 50 down_at 47 last_clean_interval [9,46) [v2:127.0.0.1:6810/2901009771,v1:127.0.0.1:6811/2901009771] [v2:127.0.0.1:6812/2901009771,v1:127.0.0.1:6813/2901009771] exists,up 0107cbfc-eed6-42f4-b707-bfb3f3155b29 2026-02-17T11:42:41.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:42:41.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:42:41.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:42:41.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:42:41.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:42:41.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:42:41.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:42:41.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:42:41.670 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:42:41.670 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:42:41.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:42:41.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:42:41.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:42:41.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:42:41.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:42:41.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:42:41.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:42:41.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:42:41.780 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:42:41.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:42:41.937 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:42:41.938 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:42:41.938 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T11:42:41.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:42:41.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:41.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:42:42.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836499 2026-02-17T11:42:42.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836499 2026-02-17T11:42:42.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499' 2026-02-17T11:42:42.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:42.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:42:42.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=214748364802 2026-02-17T11:42:42.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 214748364802 2026-02-17T11:42:42.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-214748364802' 2026-02-17T11:42:42.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:42.082 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:42:42.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574863 2026-02-17T11:42:42.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574863 2026-02-17T11:42:42.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-214748364802 2-55834574863' 2026-02-17T11:42:42.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:42.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:42:42.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561029 2026-02-17T11:42:42.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561029 2026-02-17T11:42:42.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-214748364802 2-55834574863 3-188978561029' 2026-02-17T11:42:42.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:42.227 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836499 2026-02-17T11:42:42.227 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:42.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:42:42.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836499 2026-02-17T11:42:42.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:42.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836499 2026-02-17T11:42:42.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2026-02-17T11:42:42.230 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836499 2026-02-17T11:42:42.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:42:42.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836499 2026-02-17T11:42:42.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:42.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-214748364802 2026-02-17T11:42:42.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:42.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:42:42.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-214748364802 2026-02-17T11:42:42.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:42.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=214748364802 2026-02-17T11:42:42.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 214748364802' 2026-02-17T11:42:42.394 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 214748364802 2026-02-17T11:42:42.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:42:42.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 214748364802 2026-02-17T11:42:42.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:42:43.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:42:43.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:42:43.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 214748364802 2026-02-17T11:42:43.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:42:44.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:42:44.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:42:44.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 214748364802 -lt 214748364802 2026-02-17T11:42:44.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:44.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574863 2026-02-17T11:42:44.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:44.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:42:44.871 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574863 2026-02-17T11:42:44.871 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:44.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574863 2026-02-17T11:42:44.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574863' 2026-02-17T11:42:44.873 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 55834574863 2026-02-17T11:42:44.873 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:42:45.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574864 -lt 55834574863 2026-02-17T11:42:45.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:45.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-188978561029 2026-02-17T11:42:45.027 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:45.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:42:45.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-188978561029 2026-02-17T11:42:45.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:45.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561029 2026-02-17T11:42:45.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 188978561029' 2026-02-17T11:42:45.030 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.3 seq 188978561029 2026-02-17T11:42:45.030 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:42:45.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561029 -lt 188978561029 2026-02-17T11:42:45.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:42:45.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:42:45.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:42:45.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:42:45.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:42:45.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:42:45.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:42:45.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:42:45.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:42:45.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:42:45.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:42:45.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:42:45.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:42:45.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:42:45.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:42:45.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:42:45.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:42:45.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:42:45.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-17T11:42:45.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:42:45.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:42:45.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:42:45.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:42:45.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=2.0 2026-02-17T11:42:45.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 2.0 2026-02-17T11:42:45.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T11:42:45.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T11:42:45.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:42:45.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:42:45.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:42:45.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:42:46.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:42:20.631527+0000 2026-02-17T11:42:46.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T11:42:46.157 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T11:42:46.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T11:42:20.631527+0000 2026-02-17T11:42:46.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:42:46.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:42:20.631527+0000 2026-02-17T11:42:46.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:42:46.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:42:46.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:42:46.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:42:46.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:42:46.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:42:46.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:42:46.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:42:46.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:42:20.631527+0000 '>' 2026-02-17T11:42:20.631527+0000 2026-02-17T11:42:46.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:42:47.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:42:47.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:42:47.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:42:47.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:42:47.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:42:47.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:42:47.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:42:47.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:42:20.631527+0000 '>' 2026-02-17T11:42:20.631527+0000 2026-02-17T11:42:47.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:42:48.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:42:48.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:42:48.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:42:48.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:42:48.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:42:48.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:42:48.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:42:48.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:42:20.631527+0000 '>' 2026-02-17T11:42:20.631527+0000 2026-02-17T11:42:48.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:42:49.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:42:49.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:42:49.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:42:49.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:42:49.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:42:49.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:42:49.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:42:49.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:42:20.631527+0000 '>' 2026-02-17T11:42:20.631527+0000 2026-02-17T11:42:49.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:42:50.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:42:50.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:42:50.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:42:50.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:42:50.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:42:50.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:42:50.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:42:50.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:42:46.575162+0000 '>' 2026-02-17T11:42:20.631527+0000 2026-02-17T11:42:50.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:42:50.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:42:50.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:42:50.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:42:50.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T11:42:50.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:42:50.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:42:50.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:42:50.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:42:50.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T11:42:50.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:42:50.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T11:42:50.928 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:42:50.928 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:42:50.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:42:50.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:42:50.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:42:52.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:42:52.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:42:52.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:42:52.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:42:52.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T11:42:52.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:42:52.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:42:52.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-17T11:42:52.547 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-17T11:42:52.548 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:42:52.548 INFO:tasks.workunit.client.0.trial128.stdout:hinfo_key 2026-02-17T11:42:52.548 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:42:53.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T11:42:53.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:42:53.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:42:53.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T11:42:53.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:42:53.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:42:53.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:42:53.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:42:53.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:42:53.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:42:53.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:42:53.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:42:53.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:42:53.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:42:53.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:42:53.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:42:53.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:42:53.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:42:53.088 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:42:53.088 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:42:53.088 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:42:53.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:42:53.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:42:53.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:42:53.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:42:53.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:42:53.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:42:53.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:42:53.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:42:53.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:42:53.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:42:53.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:42:53.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:42:53.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T11:42:53.090 INFO:tasks.workunit.client.0.trial128.stderr:start osd.3 2026-02-17T11:42:53.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:42:53.090 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T11:42:53.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T11:42:53.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:42:53.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:42:53.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:42:53.104 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:53.094+0000 7f09c223b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:42:53.106 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:53.094+0000 7f09c223b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:42:53.107 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:53.094+0000 7f09c223b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:42:53.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T11:42:53.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:42:53.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:42:53.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:42:53.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:42:53.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:53.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:42:53.246 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:42:53.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:53.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:42:53.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:54.341 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:54.330+0000 7f09c223b8c0 -1 Falling back to public interface 2026-02-17T11:42:54.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:54.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:54.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:42:54.395 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:42:54.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:54.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:42:54.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:55.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:55.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:55.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:42:55.545 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:42:55.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:55.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:42:55.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:56.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:56.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:56.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:42:56.693 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:42:56.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:56.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:42:56.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:42:57.352 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:42:57.342+0000 7f09c223b8c0 -1 osd.3 53 log_to_monitors true 2026-02-17T11:42:57.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:42:57.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:42:57.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:42:57.845 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:42:57.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:42:57.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:42:57.995 INFO:tasks.workunit.client.0.trial128.stderr:osd.3 up in weight 1 up_from 57 up_thru 57 down_at 54 last_clean_interval [44,53) [v2:127.0.0.1:6826/1607424879,v1:127.0.0.1:6827/1607424879] [v2:127.0.0.1:6828/1607424879,v1:127.0.0.1:6829/1607424879] exists,up bdb03fba-ae78-43d3-8520-91782827db02 2026-02-17T11:42:57.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:42:57.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:42:57.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:42:57.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:42:57.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:42:57.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:42:57.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:42:57.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:42:57.997 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:42:57.997 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:42:57.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:42:57.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:42:57.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:42:58.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:42:58.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:42:58.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:42:58.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:42:58.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:42:58.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:42:58.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:42:58.255 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:42:58.255 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:42:58.256 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T11:42:58.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:42:58.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:58.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:42:58.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836503 2026-02-17T11:42:58.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836503 2026-02-17T11:42:58.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503' 2026-02-17T11:42:58.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:58.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:42:58.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=214748364806 2026-02-17T11:42:58.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 214748364806 2026-02-17T11:42:58.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-214748364806' 2026-02-17T11:42:58.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:58.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:42:58.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574868 2026-02-17T11:42:58.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574868 2026-02-17T11:42:58.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-214748364806 2-55834574868' 2026-02-17T11:42:58.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:42:58.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:42:58.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=244813135874 2026-02-17T11:42:58.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 244813135874 2026-02-17T11:42:58.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-214748364806 2-55834574868 3-244813135874' 2026-02-17T11:42:58.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:42:58.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836503 2026-02-17T11:42:58.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:42:58.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:42:58.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836503 2026-02-17T11:42:58.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:42:58.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836503 2026-02-17T11:42:58.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836503' 2026-02-17T11:42:58.548 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836503 2026-02-17T11:42:58.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:42:58.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836502 -lt 21474836503 2026-02-17T11:42:58.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:42:59.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:42:59.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:42:59.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836502 -lt 21474836503 2026-02-17T11:42:59.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:43:00.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:43:00.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:43:01.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836504 -lt 21474836503 2026-02-17T11:43:01.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:01.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-214748364806 2026-02-17T11:43:01.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:01.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:43:01.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-214748364806 2026-02-17T11:43:01.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:01.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=214748364806 2026-02-17T11:43:01.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 214748364806' 2026-02-17T11:43:01.023 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 214748364806 2026-02-17T11:43:01.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:43:01.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 214748364806 -lt 214748364806 2026-02-17T11:43:01.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:01.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574868 2026-02-17T11:43:01.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:01.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:43:01.189 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574868 2026-02-17T11:43:01.189 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:01.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574868 2026-02-17T11:43:01.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574868' 2026-02-17T11:43:01.191 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 55834574868 2026-02-17T11:43:01.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:43:01.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574868 -lt 55834574868 2026-02-17T11:43:01.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:01.355 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-244813135874 2026-02-17T11:43:01.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:01.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:43:01.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-244813135874 2026-02-17T11:43:01.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:01.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=244813135874 2026-02-17T11:43:01.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 244813135874' 2026-02-17T11:43:01.359 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.3 seq 244813135874 2026-02-17T11:43:01.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:43:01.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135874 -lt 244813135874 2026-02-17T11:43:01.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:43:01.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:01.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:01.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:43:01.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:01.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:01.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:01.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:01.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:01.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:01.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:01.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:43:01.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:01.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:01.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:02.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:43:02.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:43:02.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:43:02.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:43:02.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:43:02.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:262: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 1 2 2026-02-17T11:43:02.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:185: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-17T11:43:02.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:186: corrupt_and_repair_two: local poolname=ecpool 2026-02-17T11:43:02.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: corrupt_and_repair_two: local first=1 2026-02-17T11:43:02.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: corrupt_and_repair_two: local second=2 2026-02-17T11:43:02.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:193: corrupt_and_repair_two: pids= 2026-02-17T11:43:02.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:43:02.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:43:02.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:43:02.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 158674"' 2026-02-17T11:43:02.045 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 158674' 2026-02-17T11:43:02.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T11:43:02.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:43:02.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:43:02.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 158676"' 2026-02-17T11:43:02.045 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 158676' 2026-02-17T11:43:02.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:43:02.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: corrupt_and_repair_two: wait_background pids 2026-02-17T11:43:02.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 158674 158676' 2026-02-17T11:43:02.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T11:43:02.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:43:02.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 158674 2026-02-17T11:43:02.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/158677: /' 2026-02-17T11:43:02.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T11:43:02.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/158679: /' 2026-02-17T11:43:04.554 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:43:04.554 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:43:04.555 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T11:43:04.555 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:43:04.555 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T11:43:04.555 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:43:04.555 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:43:04.555 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T11:43:04.555 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:43:04.555 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T11:43:04.556 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:43:04.556 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:43:04.556 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:43:04.556 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:43:04.556 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:43:04.556 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:43:04.556 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T11:43:04.556 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:43:04.556 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:43:04.556 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T11:43:04.556 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:43:04.556 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:43:04.557 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING remove 2026-02-17T11:43:04.557 INFO:tasks.workunit.client.0.trial128.stderr:158679: remove 2#2:eb822e21:::SOMETHING:head# 2026-02-17T11:43:04.557 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T11:43:04.557 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:43:04.557 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:43:04.557 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T11:43:04.557 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:43:04.557 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:43:04.557 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:43:04.557 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:43:04.558 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:43:04.559 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:43:04.559 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:43:04.559 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:43:04.559 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:43:04.559 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:43:04.559 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:43:04.559 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:43:04.559 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:43:04.559 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:43:04.559 INFO:tasks.workunit.client.0.trial128.stderr:158679: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:43:04.559 INFO:tasks.workunit.client.0.trial128.stderr:158679: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:43:04.559 INFO:tasks.workunit.client.0.trial128.stderr:158679: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:43:04.560 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:04.560 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:04.560 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:43:04.560 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:43:04.560 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:43:04.560 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:43:04.560 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:43:04.560 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:43:04.560 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:43:04.560 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:43:04.560 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:43:04.561 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:43:04.561 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T11:43:04.561 INFO:tasks.workunit.client.0.trial128.stderr:158679: start osd.2 2026-02-17T11:43:04.561 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:43:04.600 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:43:04.600 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:43:04.600 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:43:04.600 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:43:04.601 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:43:04.601 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:43:04.601 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:43:04.601 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:43:04.601 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:43:04.601 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:43:04.601 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:43:04.601 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:43:04.601 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:43:04.601 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:43:04.601 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:43:04.601 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:43:04.602 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:43:04.602 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:43:04.602 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:43:04.602 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:43:04.602 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:43:04.602 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:43:04.602 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T11:43:04.602 INFO:tasks.workunit.client.0.trial128.stderr:158677: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T11:43:04.602 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:43:04.602 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:43:04.602 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:43:04.602 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:43:04.603 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:43:04.603 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:43:04.603 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:43:04.603 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:43:04.603 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:43:04.603 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:43:04.603 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:43:04.604 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:43:04.604 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:43:04.604 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:43:04.604 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:43:04.604 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:43:04.604 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:43:04.604 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:43:04.604 INFO:tasks.workunit.client.0.trial128.stderr:158677: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:43:04.604 INFO:tasks.workunit.client.0.trial128.stderr:158677: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:43:04.604 INFO:tasks.workunit.client.0.trial128.stderr:158677: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:43:04.604 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:04.605 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:04.605 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:43:04.605 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:43:04.605 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:43:04.605 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:43:04.605 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:43:04.605 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:43:04.605 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:43:04.605 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:43:04.605 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:43:04.605 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:43:04.605 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:43:04.605 INFO:tasks.workunit.client.0.trial128.stderr:158677: start osd.1 2026-02-17T11:43:04.606 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:43:08.991 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:43:08.991 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:43:08.991 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:43:08.991 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:43:08.991 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:43:08.992 INFO:tasks.workunit.client.0.trial128.stderr:158677: 2026-02-17T11:43:04.609+0000 7f167e0468c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:08.992 INFO:tasks.workunit.client.0.trial128.stderr:158677: 2026-02-17T11:43:04.613+0000 7f167e0468c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:08.992 INFO:tasks.workunit.client.0.trial128.stderr:158677: 2026-02-17T11:43:04.613+0000 7f167e0468c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:08.992 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:43:08.992 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:43:08.992 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:43:08.992 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:43:08.992 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:43:08.992 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:08.992 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:43:08.992 INFO:tasks.workunit.client.0.trial128.stderr:158677: 0 2026-02-17T11:43:08.993 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:08.993 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:08.993 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:08.993 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:08.993 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:08.993 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:43:08.993 INFO:tasks.workunit.client.0.trial128.stderr:158677: 1 2026-02-17T11:43:08.993 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:08.993 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:08.993 INFO:tasks.workunit.client.0.trial128.stderr:158677: 2026-02-17T11:43:05.925+0000 7f167e0468c0 -1 Falling back to public interface 2026-02-17T11:43:08.993 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:08.993 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:08.993 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:08.994 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:43:08.994 INFO:tasks.workunit.client.0.trial128.stderr:158677: 2 2026-02-17T11:43:08.994 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:08.994 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:08.994 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:08.994 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:08.994 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:08.994 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:43:08.994 INFO:tasks.workunit.client.0.trial128.stderr:158677: 3 2026-02-17T11:43:08.994 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:08.994 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:08.994 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:09.156 INFO:tasks.workunit.client.0.trial128.stderr:158677: 2026-02-17T11:43:08.981+0000 7f167e0468c0 158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T11:43:09.156 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T11:43:09.156 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:43:09.156 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:43:09.156 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:43:09.156 INFO:tasks.workunit.client.0.trial128.stderr:158679: 2026-02-17T11:43:04.573+0000 7fd167b1e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:09.157 INFO:tasks.workunit.client.0.trial128.stderr:158679: 2026-02-17T11:43:04.573+0000 7fd167b1e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:09.157 INFO:tasks.workunit.client.0.trial128.stderr:158679: 2026-02-17T11:43:04.577+0000 7fd167b1e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:09.157 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T11:43:09.157 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:43:09.157 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:43:09.157 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:43:09.157 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:43:09.157 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:09.157 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:43:09.157 INFO:tasks.workunit.client.0.trial128.stderr:158679: 0 2026-02-17T11:43:09.157 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:09.157 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:43:09.158 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:09.158 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:09.158 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:09.158 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:43:09.158 INFO:tasks.workunit.client.0.trial128.stderr:158679: 1 2026-02-17T11:43:09.158 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:09.158 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:43:09.158 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:09.158 INFO:tasks.workunit.client.0.trial128.stderr:158679: 2026-02-17T11:43:06.113+0000 7fd167b1e8c0 -1 Falling back to public interface 2026-02-17T11:43:09.158 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:09.158 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:09.158 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:43:09.159 INFO:tasks.workunit.client.0.trial128.stderr:158679: 2 2026-02-17T11:43:09.159 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:09.159 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:43:09.159 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:09.159 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:09.159 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:09.159 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:43:09.159 INFO:tasks.workunit.client.0.trial128.stderr:158679: 3 2026-02-17T11:43:09.159 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:09.159 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:43:09.159 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:09.819 INFO:tasks.workunit.client.0.trial128.stderr:158679: 2026-02-17T11:43:09.145+0000 7fd167b1e8c0 -1 osd.2 59 log_to_monitors true 2026-02-17T11:43:09.820 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:09.820 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:09.820 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:43:09.820 INFO:tasks.workunit.client.0.trial128.stderr:158679: 4 2026-02-17T11:43:09.820 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:09.820 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:43:09.820 INFO:tasks.workunit.client.0.trial128.stderr:158679: osd.2 up in weight 1 up_from 65 up_thru 47 down_at 60 last_clean_interval [13,59) [v2:127.0.0.1:6818/1729732891,v1:127.0.0.1:6819/1729732891] [v2:127.0.0.1:6820/1729732891,v1:127.0.0.1:6821/1729732891] exists,up 9deb3e7d-4fba-421d-9345-6e38593da42b 2026-02-17T11:43:09.820 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:43:09.820 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:43:09.820 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:43:09.821 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:43:09.821 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:43:09.821 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:43:09.821 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:43:09.821 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:43:09.821 INFO:tasks.workunit.client.0.trial128.stderr:158679: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:43:09.821 INFO:tasks.workunit.client.0.trial128.stderr:158679: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:43:09.821 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:43:09.821 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:43:09.821 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:43:09.821 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:43:09.821 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:43:09.821 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:43:09.822 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:43:09.822 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:43:09.822 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:43:09.822 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:43:09.822 INFO:tasks.workunit.client.0.trial128.stderr:158679: 1 2026-02-17T11:43:09.822 INFO:tasks.workunit.client.0.trial128.stderr:158679: 2 2026-02-17T11:43:09.822 INFO:tasks.workunit.client.0.trial128.stderr:158679: 3' 2026-02-17T11:43:09.822 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:43:09.822 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:09.822 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:43:09.822 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836507 2026-02-17T11:43:09.822 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836507 2026-02-17T11:43:09.823 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507' 2026-02-17T11:43:09.823 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:09.856 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-hel-1 osd.1 59 log_to_monitors true 2026-02-17T11:43:09.857 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:09.857 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:09.857 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:43:09.857 INFO:tasks.workunit.client.0.trial128.stderr:158677: 4 2026-02-17T11:43:09.857 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:09.857 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:09.857 INFO:tasks.workunit.client.0.trial128.stderr:158677: osd.1 up in weight 1 up_from 63 up_thru 65 down_at 60 last_clean_interval [50,59) [v2:127.0.0.1:6810/4029400380,v1:127.0.0.1:6811/4029400380] [v2:127.0.0.1:6812/4029400380,v1:127.0.0.1:6813/4029400380] exists,up 0107cbfc-eed6-42f4-b707-bfb3f3155b29 2026-02-17T11:43:09.857 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:43:09.857 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:43:09.858 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:43:09.858 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:43:09.858 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:43:09.858 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:43:09.858 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:43:09.858 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:43:09.858 INFO:tasks.workunit.client.0.trial128.stderr:158677: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:43:09.858 INFO:tasks.workunit.client.0.trial128.stderr:158677: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:43:09.858 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:43:09.858 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:43:09.858 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:43:09.858 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:43:09.859 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:43:09.859 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:43:09.859 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:43:09.859 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:43:09.859 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:43:09.859 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:43:09.859 INFO:tasks.workunit.client.0.trial128.stderr:158677: 1 2026-02-17T11:43:09.859 INFO:tasks.workunit.client.0.trial128.stderr:158677: 2 2026-02-17T11:43:09.859 INFO:tasks.workunit.client.0.trial128.stderr:158677: 3' 2026-02-17T11:43:09.859 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:43:09.859 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:09.859 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:43:09.860 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836508 2026-02-17T11:43:09.860 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836508 2026-02-17T11:43:09.860 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508' 2026-02-17T11:43:09.860 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:10.355 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:43:10.355 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=270582939650 2026-02-17T11:43:10.355 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 270582939650 2026-02-17T11:43:10.355 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-270582939650' 2026-02-17T11:43:10.355 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:10.355 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:43:10.355 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874242 2026-02-17T11:43:10.355 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874242 2026-02-17T11:43:10.356 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-270582939650 2-279172874242' 2026-02-17T11:43:10.356 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:10.356 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:43:10.356 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=244813135877 2026-02-17T11:43:10.356 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 244813135877 2026-02-17T11:43:10.356 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-270582939650 2-279172874242 3-244813135877' 2026-02-17T11:43:10.356 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:10.356 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836507 2026-02-17T11:43:10.356 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:10.356 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:43:10.356 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836507 2026-02-17T11:43:10.356 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:10.357 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836507 2026-02-17T11:43:10.357 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836507' 2026-02-17T11:43:10.357 INFO:tasks.workunit.client.0.trial128.stderr:158679: waiting osd.0 seq 21474836507 2026-02-17T11:43:10.357 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:43:10.357 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836508 -lt 21474836507 2026-02-17T11:43:10.357 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:10.357 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-270582939650 2026-02-17T11:43:10.357 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:10.357 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:43:10.357 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-270582939650 2026-02-17T11:43:10.357 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:10.357 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=270582939650 2026-02-17T11:43:10.358 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 270582939650' 2026-02-17T11:43:10.358 INFO:tasks.workunit.client.0.trial128.stderr:158679: waiting osd.1 seq 270582939650 2026-02-17T11:43:10.358 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:43:10.378 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:227pers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:43:10.378 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=270582939651 2026-02-17T11:43:10.378 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 270582939651 2026-02-17T11:43:10.379 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-270582939651' 2026-02-17T11:43:10.379 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:10.379 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:43:10.379 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874243 2026-02-17T11:43:10.379 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874243 2026-02-17T11:43:10.379 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-270582939651 2-279172874243' 2026-02-17T11:43:10.379 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:10.379 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:43:10.379 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=244813135878 2026-02-17T11:43:10.379 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 244813135878 2026-02-17T11:43:10.379 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-270582939651 2-279172874243 3-244813135878' 2026-02-17T11:43:10.380 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:10.380 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836508 2026-02-17T11:43:10.380 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:10.380 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:43:10.380 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836508 2026-02-17T11:43:10.380 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:10.380 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836508 2026-02-17T11:43:10.380 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836508' 2026-02-17T11:43:10.380 INFO:tasks.workunit.client.0.trial128.stderr:158677: waiting osd.0 seq 21474836508 2026-02-17T11:43:10.380 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:43:10.380 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836508 -lt 21474836508 2026-02-17T11:43:10.380 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:10.381 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-270582939651 2026-02-17T11:43:10.381 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:10.381 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:43:10.381 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-270582939651 2026-02-17T11:43:10.381 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:10.381 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=270582939651 2026-02-17T11:43:10.381 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 270582939651' 2026-02-17T11:43:10.381 INFO:tasks.workunit.client.0.trial128.stderr:158677: waiting osd.1 seq 270582939651 2026-02-17T11:43:10.381 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:43:13.174 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 270582939651 -lt 270582939651 2026-02-17T11:43:13.174 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:13.175 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-279172874243 2026-02-17T11:43:13.175 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:13.175 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:43:13.175 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-279172874243 2026-02-17T11:43:13.175 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:13.175 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874243 2026-02-17T11:43:13.175 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 279172874243' 2026-02-17T11:43:13.175 INFO:tasks.workunit.client.0.trial128.stderr:158677: waiting osd.2 seq 279172874243 2026-02-17T11:43:13.175 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:43:13.175 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874243 -lt 279172874243 2026-02-17T11:43:13.175 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:13.176 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-244813135878 2026-02-17T11:43:13.176 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:13.176 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:43:13.176 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-244813135878 2026-02-17T11:43:13.176 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:13.176 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=244813135878 2026-02-17T11:43:13.176 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 244813135878' 2026-02-17T11:43:13.176 INFO:tasks.workunit.client.0.trial128.stderr:158677: waiting osd.3 seq 244813135878 2026-02-17T11:43:13.176 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:43:13.176 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135876 -lt 244813135878 2026-02-17T11:43:13.176 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:43:13.176 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:43:13.177 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:43:13.177 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135876 -lt 244813135878 2026-02-17T11:43:13.177 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:43:13.177 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:43:13.177 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:43:13.177 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135878 -lt 244813135878 2026-02-17T11:43:13.177 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:43:13.177 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:13.177 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:13.177 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:43:13.177 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:13.186 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-he7: flush_pg_stats: test 270582939651 -lt 270582939650 2026-02-17T11:43:13.187 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:13.187 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-279172874242 2026-02-17T11:43:13.187 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:13.187 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:43:13.187 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-279172874242 2026-02-17T11:43:13.187 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:13.187 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874242 2026-02-17T11:43:13.187 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 279172874242' 2026-02-17T11:43:13.187 INFO:tasks.workunit.client.0.trial128.stderr:158679: waiting osd.2 seq 279172874242 2026-02-17T11:43:13.187 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:43:13.187 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874243 -lt 279172874242 2026-02-17T11:43:13.188 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:13.188 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-244813135877 2026-02-17T11:43:13.188 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:13.188 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:43:13.188 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-244813135877 2026-02-17T11:43:13.188 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:13.188 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=244813135877 2026-02-17T11:43:13.188 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 244813135877' 2026-02-17T11:43:13.188 INFO:tasks.workunit.client.0.trial128.stderr:158679: waiting osd.3 seq 244813135877 2026-02-17T11:43:13.188 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:43:13.188 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135876 -lt 244813135877 2026-02-17T11:43:13.188 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:43:13.189 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:43:13.189 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:43:13.189 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135876 -lt 244813135877 2026-02-17T11:43:13.189 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:43:13.189 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:43:13.189 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:43:13.189 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135878 -lt 244813135877 2026-02-17T11:43:13.189 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:43:13.189 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:13.189 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:13.189 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:43:13.189 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:13.937 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:13.938 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:13.938 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:13.938 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:13.938 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:13.938 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:13.938 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:43:13.938 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:13.938 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:13.938 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:13.938 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 5 2026-02-17T11:43:13.939 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 1 '!=' -1 2026-02-17T11:43:13.939 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T11:43:13.939 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=1 2026-02-17T11:43:13.939 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:43:13.939 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T11:43:13.939 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:43:13.939 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:13.939 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:13.939 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:13.939 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:13.939 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:13.939 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:13.939 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:13.940 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:43:13.940 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:13.940 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:13.940 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:13.940 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 5 2026-02-17T11:43:13.940 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 1 '!=' 1 2026-02-17T11:43:13.940 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:43:13.940 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:43:13.973 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntlpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:13.973 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:13.973 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:13.973 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:13.973 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:13.973 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:13.974 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:43:13.974 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:13.974 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:13.974 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:13.974 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 5 2026-02-17T11:43:13.974 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 1 '!=' -1 2026-02-17T11:43:13.974 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T11:43:13.974 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=1 2026-02-17T11:43:13.974 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:43:13.974 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T11:43:13.974 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:43:13.974 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:13.975 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:13.975 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:13.975 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:13.975 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:13.975 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:13.975 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:13.975 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T11:43:13.975 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:13.975 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:13.975 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:13.975 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 5 2026-02-17T11:43:13.975 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 1 '!=' 1 2026-02-17T11:43:13.976 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:43:13.976 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:43:14.751 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:43:14.752 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:43:14.752 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:43:14.752 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:43:14.752 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:43:14.752 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:43:14.752 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:43:14.752 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T11:43:14.752 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:43:14.752 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T11:43:14.753 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:43:14.753 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:14.753 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:14.753 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:14.753 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:14.753 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:14.753 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:14.753 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:14.753 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T11:43:14.753 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:14.753 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:14.753 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:14.754 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T11:43:14.754 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 1 2026-02-17T11:43:14.754 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T11:43:14.754 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=3 2026-02-17T11:43:14.754 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:43:14.754 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T11:43:14.754 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:43:14.754 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:14.754 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:14.804 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: geu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:43:14.804 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:43:14.805 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:43:14.805 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:43:14.805 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:43:14.805 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:43:14.805 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:43:14.805 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T11:43:14.805 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:43:14.805 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T11:43:14.805 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:43:14.805 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:14.805 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:14.806 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:14.806 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:14.806 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:14.806 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:14.806 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:14.806 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T11:43:14.806 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:14.806 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:14.806 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:14.806 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T11:43:14.806 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 1 2026-02-17T11:43:14.806 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T11:43:14.806 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=3 2026-02-17T11:43:14.807 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:43:14.807 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T11:43:14.807 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:43:14.807 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:14.807 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:15.474 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:15.475 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:15.475 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:15.475 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:15.475 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:15.475 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T11:43:15.475 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:15.475 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:15.475 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:15.475 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T11:43:15.475 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T11:43:15.475 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:43:15.476 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:43:15.476 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:43:15.476 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:43:15.476 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:43:15.476 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:43:15.476 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:43:15.476 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:43:15.476 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:43:15.476 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T11:43:15.476 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:43:15.476 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T11:43:15.477 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:43:15.477 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:15.477 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:15.477 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:15.477 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:15.477 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:15.477 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:15.532 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/cet_num_active_clean: local expression 2026-02-17T11:43:15.533 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:15.533 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:15.533 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:15.533 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:15.533 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T11:43:15.533 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:15.533 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:15.533 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:15.533 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T11:43:15.533 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T11:43:15.534 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:43:15.534 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:43:15.534 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:43:15.534 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:43:15.534 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:43:15.534 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:43:15.534 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:43:15.534 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:43:15.534 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:43:15.534 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T11:43:15.534 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:43:15.534 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T11:43:15.535 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:43:15.535 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:15.535 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:15.535 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:15.535 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:15.535 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:15.535 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:16.571 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:16.572 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T11:43:16.572 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:16.572 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:16.572 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:16.572 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T11:43:16.572 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T11:43:16.572 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:43:16.572 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:43:16.572 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:43:16.572 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:43:16.573 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:43:16.573 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:43:16.573 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:43:16.573 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:43:16.573 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:43:16.573 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T11:43:16.573 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:43:16.573 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T11:43:16.573 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:43:16.573 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:16.573 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:16.573 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:16.574 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:16.574 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:16.574 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:16.574 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:16.574 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:43:16.574 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:16.574 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:16.590 INFO:tasks.workunit.client.0.trial128.stderr:158677: //home/ubuntu/cephtest/clone.clph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:16.590 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T11:43:16.590 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:16.591 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:16.591 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:16.591 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T11:43:16.591 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T11:43:16.591 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:43:16.591 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:43:16.591 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:43:16.591 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:43:16.591 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:43:16.591 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:43:16.591 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:43:16.591 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:43:16.592 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:43:16.592 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T11:43:16.592 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:43:16.592 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T11:43:16.592 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:43:16.592 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:16.592 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:16.592 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:16.592 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:16.592 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:16.592 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:16.592 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:16.592 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:43:16.593 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:16.593 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:16.761 INFO:tasks.workunit.client.0.trial128.stderr:158679: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:16.761 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:43:16.761 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:43:16.761 INFO:tasks.workunit.client.0.trial128.stderr:158677: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:43:16.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:43:16.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:43:16.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 158676 2026-02-17T11:43:16.766 INFO:tasks.workunit.client.0.trial128.stderr:ient.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:16.767 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:43:16.767 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:43:16.767 INFO:tasks.workunit.client.0.trial128.stderr:158679: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:43:16.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:43:16.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T11:43:16.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T11:43:16.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T11:43:16.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: corrupt_and_repair_two: return_code=0 2026-02-17T11:43:16.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:198: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T11:43:16.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-17T11:43:16.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:43:16.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:43:16.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:43:16.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:43:16.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: local pg=2.0 2026-02-17T11:43:16.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:204: corrupt_and_repair_two: repair 2.0 2026-02-17T11:43:16.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T11:43:16.925 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T11:43:16.925 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:43:16.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:43:16.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:43:16.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:43:17.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:42:46.575162+0000 2026-02-17T11:43:17.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T11:43:17.221 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T11:43:17.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T11:42:46.575162+0000 2026-02-17T11:43:17.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:43:17.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:42:46.575162+0000 2026-02-17T11:43:17.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:43:17.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:43:17.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:43:17.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:43:17.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:43:17.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:43:17.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:43:17.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:43:17.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:42:46.575162+0000 '>' 2026-02-17T11:42:46.575162+0000 2026-02-17T11:43:17.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:43:18.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:43:18.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:43:18.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:43:18.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:43:18.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:43:18.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:43:18.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:43:18.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:43:17.336523+0000 '>' 2026-02-17T11:42:46.575162+0000 2026-02-17T11:43:18.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:43:18.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: corrupt_and_repair_two: pids= 2026-02-17T11:43:18.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:43:18.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:43:18.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:43:18.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 162235"' 2026-02-17T11:43:18.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 162235' 2026-02-17T11:43:18.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-17T11:43:18.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:43:18.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:43:18.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 162237"' 2026-02-17T11:43:18.541 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 162237' 2026-02-17T11:43:18.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:43:18.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:211: corrupt_and_repair_two: wait_background pids 2026-02-17T11:43:18.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 162235 162237' 2026-02-17T11:43:18.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T11:43:18.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:43:18.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 162235 2026-02-17T11:43:18.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/162238: /' 2026-02-17T11:43:18.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-17T11:43:18.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/162240: /' 2026-02-17T11:43:20.694 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:43:20.694 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:43:20.694 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T11:43:20.694 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:43:20.694 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-17T11:43:20.694 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:43:20.694 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:43:20.694 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T11:43:20.694 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:43:20.695 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T11:43:20.695 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:43:20.695 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:43:20.695 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:43:20.695 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:43:20.695 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:43:20.695 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:43:20.695 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING list-attrs 2026-02-17T11:43:20.695 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:43:20.695 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:43:20.695 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T11:43:20.695 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:43:20.696 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:43:20.696 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING list-attrs 2026-02-17T11:43:20.696 INFO:tasks.workunit.client.0.trial128.stderr:162240: _ 2026-02-17T11:43:20.696 INFO:tasks.workunit.client.0.trial128.stderr:162240: hinfo_key 2026-02-17T11:43:20.696 INFO:tasks.workunit.client.0.trial128.stderr:162240: snapset 2026-02-17T11:43:20.696 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T11:43:20.696 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:43:20.696 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:43:20.696 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T11:43:20.696 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:43:20.696 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:43:20.696 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:43:20.697 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:43:20.697 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helper162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:43:20.697 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:43:20.698 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:43:20.698 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:43:20.698 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:43:20.698 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:43:20.698 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:43:20.698 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:43:20.698 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:43:20.698 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:43:20.698 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:43:20.698 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:43:20.698 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:43:20.699 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:43:20.699 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:43:20.699 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:43:20.699 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:43:20.699 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:43:20.699 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:43:20.699 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:43:20.699 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:43:20.699 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:43:20.699 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T11:43:20.699 INFO:tasks.workunit.client.0.trial128.stderr:162238: _ 2026-02-17T11:43:20.699 INFO:tasks.workunit.client.0.trial128.stderr:162238: hinfo_key 2026-02-17T11:43:20.700 INFO:tasks.workunit.client.0.trial128.stderr:162238: snapset 2026-02-17T11:43:20.700 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:43:20.700 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:43:20.700 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:43:20.700 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:43:20.700 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:43:20.700 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:43:20.700 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:43:20.700 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:43:20.701 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:43:20.701 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:43:20.701 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:43:20.701 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:43:20.701 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:43:20.701 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:43:20.701 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:43:20.701 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:43:20.701 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:43:20.701 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:43:20.701 INFO:tasks.workunit.client.0.trial128.stderr:162240: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:43:20.701 INFO:tasks.workunit.client.0.trial128.stderr:162240: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:43:20.702 INFO:tasks.workunit.client.0.trial128.stderr:162240: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:43:20.702 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:20.702 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:20.702 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:43:20.702 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:43:20.702 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:43:20.702 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:43:20.702 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:43:20.702 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:43:20.702 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:43:20.702 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:43:20.702 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:43:20.702 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:43:20.702 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T11:43:20.703 INFO:tasks.workunit.client.0.trial128.stderr:162240: start osd.2 2026-02-17T11:43:20.703 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recs.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:43:20.703 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:43:20.703 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:43:20.703 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:43:20.704 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:43:20.704 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:43:20.704 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:43:20.704 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:43:20.704 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:43:20.704 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:43:20.704 INFO:tasks.workunit.client.0.trial128.stderr:162238: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:43:20.704 INFO:tasks.workunit.client.0.trial128.stderr:162238: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:43:20.704 INFO:tasks.workunit.client.0.trial128.stderr:162238: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:43:20.704 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:20.704 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:20.704 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:43:20.704 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:43:20.705 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:43:20.705 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:43:20.705 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:43:20.705 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:43:20.705 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:43:20.705 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:43:20.705 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:43:20.705 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:43:20.705 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:43:20.705 INFO:tasks.workunit.client.0.trial128.stderr:162238: start osd.1 2026-02-17T11:43:25.177 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:43:25.177 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T11:43:25.177 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T11:43:25.177 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:43:25.178 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:43:25.178 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:43:25.178 INFO:tasks.workunit.client.0.trial128.stderr:162240: 2026-02-17T11:43:20.701+0000 7f99eb6f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:25.178 INFO:tasks.workunit.client.0.trial128.stderr:162240: 2026-02-17T11:43:20.705+0000 7f99eb6f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:25.178 INFO:tasks.workunit.client.0.trial128.stderr:162240: 2026-02-17T11:43:20.705+0000 7f99eb6f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:25.178 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T11:43:25.178 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:43:25.178 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:43:25.178 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:43:25.178 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:43:25.178 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:25.178 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:43:25.179 INFO:tasks.workunit.client.0.trial128.stderr:162240: 0 2026-02-17T11:43:25.179 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:25.179 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:43:25.179 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:25.179 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:25.179 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:25.179 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:43:25.179 INFO:tasks.workunit.client.0.trial128.stderr:162240: 1 2026-02-17T11:43:25.179 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:25.179 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:43:25.179 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:25.180 INFO:tasks.workunit.client.0.trial128.stderr:162240: 2026-02-17T11:43:22.177+0000 7f99eb6f38c0 -1 Falling back to public interface 2026-02-17T11:43:25.180 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:25.180 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:25.180 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:43:25.180 INFO:tasks.workunit.client.0.trial128.stderr:162240: 2 2026-02-17T11:43:25.180 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:25.180 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:43:25.180 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:25.180 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:25.180 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:25.180 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:43:25.180 INFO:tasks.workunit.client.0.trial128.stderr:162240: 3 2026-02-17T11:43:25.181 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:25.181 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:43:25.181 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:25.288 INFO:tasks.workunit.client.0.trial128.stderr:162240: 2026-02-17T11:43:25.169+0000 7f9overy_ops 2026-02-17T11:43:25.289 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:43:25.289 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:43:25.289 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:43:25.289 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:43:25.289 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:43:25.289 INFO:tasks.workunit.client.0.trial128.stderr:162238: 2026-02-17T11:43:20.701+0000 7ff831b728c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:25.289 INFO:tasks.workunit.client.0.trial128.stderr:162238: 2026-02-17T11:43:20.705+0000 7ff831b728c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:25.289 INFO:tasks.workunit.client.0.trial128.stderr:162238: 2026-02-17T11:43:20.705+0000 7ff831b728c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:25.289 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:43:25.290 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:43:25.290 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:43:25.290 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:43:25.290 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:43:25.290 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:25.290 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:43:25.290 INFO:tasks.workunit.client.0.trial128.stderr:162238: 0 2026-02-17T11:43:25.290 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:25.290 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:25.290 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:25.290 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:25.291 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:25.291 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:43:25.291 INFO:tasks.workunit.client.0.trial128.stderr:162238: 1 2026-02-17T11:43:25.291 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:25.291 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:25.291 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:25.291 INFO:tasks.workunit.client.0.trial128.stderr:162238: 2026-02-17T11:43:22.205+0000 7ff831b728c0 -1 Falling back to public interface 2026-02-17T11:43:25.291 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:25.291 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:25.291 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:43:25.292 INFO:tasks.workunit.client.0.trial128.stderr:162238: 2 2026-02-17T11:43:25.292 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:25.292 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:25.292 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:25.292 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:25.292 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:25.292 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:43:25.292 INFO:tasks.workunit.client.0.trial128.stderr:162238: 3 2026-02-17T11:43:25.292 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:25.292 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:25.292 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:25.960 INFO:tasks.workunit.client.0.trial128.stderr:162238: 2026-02-17T11:43:25.281+0000 7ff9eb6f38c0 -1 osd.2 67 log_to_monitors true 2026-02-17T11:43:25.960 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:25.960 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:25.960 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:43:25.960 INFO:tasks.workunit.client.0.trial128.stderr:162240: 4 2026-02-17T11:43:25.960 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:25.960 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:43:25.961 INFO:tasks.workunit.client.0.trial128.stderr:162240: osd.2 up in weight 1 up_from 71 up_thru 71 down_at 68 last_clean_interval [65,67) [v2:127.0.0.1:6810/2311315750,v1:127.0.0.1:6811/2311315750] [v2:127.0.0.1:6812/2311315750,v1:127.0.0.1:6813/2311315750] exists,up 9deb3e7d-4fba-421d-9345-6e38593da42b 2026-02-17T11:43:25.961 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:43:25.961 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:43:25.961 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:43:25.961 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:43:25.961 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:43:25.961 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:43:25.961 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:43:25.961 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:43:25.961 INFO:tasks.workunit.client.0.trial128.stderr:162240: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:43:25.962 INFO:tasks.workunit.client.0.trial128.stderr:162240: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:43:25.962 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:43:25.962 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:43:25.962 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:43:25.962 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:43:25.962 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:43:25.962 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:43:25.962 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:43:25.962 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:43:25.962 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:43:25.962 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:43:25.963 INFO:tasks.workunit.client.0.trial128.stderr:162240: 1 2026-02-17T11:43:25.963 INFO:tasks.workunit.client.0.trial128.stderr:162240: 2 2026-02-17T11:43:25.963 INFO:tasks.workunit.client.0.trial128.stderr:162240: 3' 2026-02-17T11:43:25.963 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:43:25.963 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:25.963 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:43:25.963 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836512 2026-02-17T11:43:25.963 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836512 2026-02-17T11:43:25.963 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836512' 2026-02-17T11:43:25.963 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:25.981 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalon831b728c0 -1 osd.1 67 log_to_monitors true 2026-02-17T11:43:25.981 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:25.981 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:25.981 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:43:25.981 INFO:tasks.workunit.client.0.trial128.stderr:162238: 4 2026-02-17T11:43:25.981 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:25.982 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:25.982 INFO:tasks.workunit.client.0.trial128.stderr:162238: osd.1 up in weight 1 up_from 72 up_thru 72 down_at 68 last_clean_interval [63,67) [v2:127.0.0.1:6818/919056762,v1:127.0.0.1:6819/919056762] [v2:127.0.0.1:6820/919056762,v1:127.0.0.1:6821/919056762] exists,up 0107cbfc-eed6-42f4-b707-bfb3f3155b29 2026-02-17T11:43:25.982 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:43:25.982 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:43:25.982 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:43:25.982 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:43:25.982 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:43:25.982 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:43:25.983 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:43:25.983 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:43:25.983 INFO:tasks.workunit.client.0.trial128.stderr:162238: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:43:25.983 INFO:tasks.workunit.client.0.trial128.stderr:162238: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:43:25.983 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:43:25.983 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:43:25.983 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:43:25.983 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:43:25.983 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:43:25.983 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:43:25.983 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:43:25.983 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:43:25.984 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:43:25.984 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:43:25.984 INFO:tasks.workunit.client.0.trial128.stderr:162238: 1 2026-02-17T11:43:25.984 INFO:tasks.workunit.client.0.trial128.stderr:162238: 2 2026-02-17T11:43:25.984 INFO:tasks.workunit.client.0.trial128.stderr:162238: 3' 2026-02-17T11:43:25.984 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:43:25.984 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:25.984 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:43:25.984 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836513 2026-02-17T11:43:25.984 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836513 2026-02-17T11:43:25.984 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513' 2026-02-17T11:43:25.985 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:26.508 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/cee/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:43:26.508 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=309237645314 2026-02-17T11:43:26.508 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 309237645314 2026-02-17T11:43:26.509 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836512 1-309237645314' 2026-02-17T11:43:26.509 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:26.509 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:43:26.509 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678018 2026-02-17T11:43:26.509 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678018 2026-02-17T11:43:26.509 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836512 1-309237645314 2-304942678018' 2026-02-17T11:43:26.509 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:26.509 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:43:26.509 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=244813135882 2026-02-17T11:43:26.510 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 244813135882 2026-02-17T11:43:26.510 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836512 1-309237645314 2-304942678018 3-244813135882' 2026-02-17T11:43:26.510 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:26.510 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836512 2026-02-17T11:43:26.510 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:26.510 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:43:26.510 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836512 2026-02-17T11:43:26.510 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:26.510 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836512 2026-02-17T11:43:26.510 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836512' 2026-02-17T11:43:26.510 INFO:tasks.workunit.client.0.trial128.stderr:162240: waiting osd.0 seq 21474836512 2026-02-17T11:43:26.511 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:43:26.511 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836513 -lt 21474836512 2026-02-17T11:43:26.511 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:26.511 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-309237645314 2026-02-17T11:43:26.511 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:26.511 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:43:26.511 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-309237645314 2026-02-17T11:43:26.511 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:26.511 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=309237645314 2026-02-17T11:43:26.511 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 309237645314' 2026-02-17T11:43:26.511 INFO:tasks.workunit.client.0.trial128.stderr:162240: waiting osd.1 seq 309237645314 2026-02-17T11:43:26.511 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:43:26.512 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:43:26.512 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=309237645315 2026-02-17T11:43:26.512 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 309237645315 2026-02-17T11:43:26.512 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-309237645315' 2026-02-17T11:43:26.512 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:26.513 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:43:26.513 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678019 2026-02-17T11:43:26.513 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678019 2026-02-17T11:43:26.513 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-309237645315 2-304942678019' 2026-02-17T11:43:26.513 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:26.513 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:43:26.513 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=244813135883 2026-02-17T11:43:26.513 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 244813135883 2026-02-17T11:43:26.513 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-309237645315 2-304942678019 3-244813135883' 2026-02-17T11:43:26.513 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:26.513 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836513 2026-02-17T11:43:26.513 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:26.514 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:43:26.514 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836513 2026-02-17T11:43:26.514 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:26.514 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836513 2026-02-17T11:43:26.514 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836513' 2026-02-17T11:43:26.514 INFO:tasks.workunit.client.0.trial128.stderr:162238: waiting osd.0 seq 21474836513 2026-02-17T11:43:26.514 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:43:26.514 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836513 -lt 21474836513 2026-02-17T11:43:26.514 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:26.514 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-309237645315 2026-02-17T11:43:26.514 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:26.515 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:43:26.515 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-309237645315 2026-02-17T11:43:26.515 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:26.515 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=309237645315 2026-02-17T11:43:26.515 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 309237645315' 2026-02-17T11:43:26.515 INFO:tasks.workunit.client.0.trial128.stderr:162238: waiting osd.1 seq 309237645315 2026-02-17T11:43:26.515 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:43:29.312 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.ers.sh:2277: flush_pg_stats: test 1 -lt 309237645314 2026-02-17T11:43:29.312 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:43:29.312 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:43:29.312 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:43:29.313 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 309237645314 2026-02-17T11:43:29.313 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:43:29.313 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:43:29.313 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:43:29.313 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 309237645315 -lt 309237645314 2026-02-17T11:43:29.313 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:29.313 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-304942678018 2026-02-17T11:43:29.313 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:29.313 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:43:29.313 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-304942678018 2026-02-17T11:43:29.313 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:29.314 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678018 2026-02-17T11:43:29.314 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 304942678018' 2026-02-17T11:43:29.314 INFO:tasks.workunit.client.0.trial128.stderr:162240: waiting osd.2 seq 304942678018 2026-02-17T11:43:29.314 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:43:29.314 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678019 -lt 304942678018 2026-02-17T11:43:29.314 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:29.314 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-244813135882 2026-02-17T11:43:29.314 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:29.314 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:43:29.314 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-244813135882 2026-02-17T11:43:29.314 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:29.314 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=244813135882 2026-02-17T11:43:29.315 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 244813135882' 2026-02-17T11:43:29.315 INFO:tasks.workunit.client.0.trial128.stderr:162240: waiting osd.3 seq 244813135882 2026-02-17T11:43:29.315 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:43:29.315 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135884 -lt 244813135882 2026-02-17T11:43:29.315 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:43:29.315 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:29.315 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:29.315 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:43:29.315 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:29.335 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167sh:2277: flush_pg_stats: test 1 -lt 309237645315 2026-02-17T11:43:29.335 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:43:29.335 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:43:29.335 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:43:29.335 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 309237645315 2026-02-17T11:43:29.335 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:43:29.335 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:43:29.335 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:43:29.335 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 309237645315 -lt 309237645315 2026-02-17T11:43:29.336 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:29.336 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-304942678019 2026-02-17T11:43:29.336 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:29.336 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:43:29.336 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-304942678019 2026-02-17T11:43:29.336 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:29.336 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678019 2026-02-17T11:43:29.336 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 304942678019' 2026-02-17T11:43:29.336 INFO:tasks.workunit.client.0.trial128.stderr:162238: waiting osd.2 seq 304942678019 2026-02-17T11:43:29.336 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:43:29.336 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678019 -lt 304942678019 2026-02-17T11:43:29.336 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:29.337 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-244813135883 2026-02-17T11:43:29.337 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:29.337 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:43:29.337 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-244813135883 2026-02-17T11:43:29.337 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:29.337 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=244813135883 2026-02-17T11:43:29.337 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 244813135883' 2026-02-17T11:43:29.337 INFO:tasks.workunit.client.0.trial128.stderr:162238: waiting osd.3 seq 244813135883 2026-02-17T11:43:29.337 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:43:29.337 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 244813135884 -lt 244813135883 2026-02-17T11:43:29.337 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:43:29.338 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:29.338 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:29.338 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:43:29.338 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:29.649 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: w1: wait_for_clean: get_num_active_clean 2026-02-17T11:43:29.649 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:29.649 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:29.649 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:29.649 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:29.649 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:29.649 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:43:29.649 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:29.649 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:29.650 INFO:tasks.workunit.client.0.trial128.stderr:162240: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:29.650 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:43:29.650 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:43:29.650 INFO:tasks.workunit.client.0.trial128.stderr:162240: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:43:29.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:43:29.679 INFO:tasks.workunit.client.0.trial128.stderr:ait_for_clean: get_num_active_clean 2026-02-17T11:43:29.679 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:29.679 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:29.680 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:29.680 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:29.680 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:29.680 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:43:29.680 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:29.680 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:29.680 INFO:tasks.workunit.client.0.trial128.stderr:162238: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:29.680 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:43:29.680 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:43:29.680 INFO:tasks.workunit.client.0.trial128.stderr:162238: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:43:29.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:43:29.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:43:29.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 162237 2026-02-17T11:43:29.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T11:43:29.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T11:43:29.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T11:43:29.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:212: corrupt_and_repair_two: return_code=0 2026-02-17T11:43:29.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:213: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T11:43:29.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:43:29.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:43:29.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 3 1 2026-02-17T11:43:29.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:185: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-17T11:43:29.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:186: corrupt_and_repair_two: local poolname=ecpool 2026-02-17T11:43:29.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: corrupt_and_repair_two: local first=3 2026-02-17T11:43:29.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: corrupt_and_repair_two: local second=1 2026-02-17T11:43:29.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:193: corrupt_and_repair_two: pids= 2026-02-17T11:43:29.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:43:29.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:43:29.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:43:29.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 164698"' 2026-02-17T11:43:29.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 164698' 2026-02-17T11:43:29.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:43:29.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:43:29.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:43:29.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 164700"' 2026-02-17T11:43:29.703 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 164700' 2026-02-17T11:43:29.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:43:29.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: corrupt_and_repair_two: wait_background pids 2026-02-17T11:43:29.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 164698 164700' 2026-02-17T11:43:29.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T11:43:29.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:43:29.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 164698 2026-02-17T11:43:29.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/164701: /' 2026-02-17T11:43:29.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:43:29.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/164704: /' 2026-02-17T11:43:32.188 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:43:32.188 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:43:32.188 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:43:32.188 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:43:32.188 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:43:32.188 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:43:32.188 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:43:32.188 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:43:32.189 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:43:32.190 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:43:32.190 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:43:32.191 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:43:32.192 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:43:32.193 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:43:32.193 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:43:32.193 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:43:32.193 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:43:32.193 INFO:tasks.workunit.client.0.trial128.stderr:164704: start osd.1 2026-02-17T11:43:32.193 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:43:32.273 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:43:32.273 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:43:32.273 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T11:43:32.273 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:43:32.273 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:43:32.273 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:43:32.273 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:43:32.274 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T11:43:32.274 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:43:32.274 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T11:43:32.274 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:43:32.274 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:43:32.274 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:43:32.274 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:43:32.274 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:43:32.274 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:43:32.274 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:43:32.274 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:43:32.274 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:43:32.275 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T11:43:32.275 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:43:32.275 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:43:32.275 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T11:43:32.275 INFO:tasks.workunit.client.0.trial128.stderr:164701: remove 0#2:eb822e21:::SOMETHING:head# 2026-02-17T11:43:32.275 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T11:43:32.275 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:43:32.275 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:43:32.275 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T11:43:32.275 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:43:32.275 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:43:32.276 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:43:32.276 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:43:32.276 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:43:32.276 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:43:32.276 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:43:32.277 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:43:32.277 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:43:32.277 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:43:32.277 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:43:32.277 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:43:32.277 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:43:32.277 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:43:32.277 INFO:tasks.workunit.client.0.trial128.stderr:164701: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:43:32.277 INFO:tasks.workunit.client.0.trial128.stderr:164701: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:43:32.277 INFO:tasks.workunit.client.0.trial128.stderr:164701: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:43:32.277 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:32.277 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:32.278 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:43:32.278 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:43:32.278 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:43:32.278 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:43:32.278 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:43:32.278 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:43:32.278 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:43:32.278 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:43:32.278 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:43:32.278 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:43:32.278 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T11:43:32.278 INFO:tasks.workunit.client.0.trial128.stderr:164701: start osd.3 2026-02-17T11:43:32.278 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:43:36.599 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:43:36.599 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:43:36.599 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:43:36.599 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:43:36.600 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:43:36.600 INFO:tasks.workunit.client.0.trial128.stderr:164704: 2026-02-17T11:43:32.201+0000 7fa09f8628c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:36.600 INFO:tasks.workunit.client.0.trial128.stderr:164704: 2026-02-17T11:43:32.201+0000 7fa09f8628c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:36.600 INFO:tasks.workunit.client.0.trial128.stderr:164704: 2026-02-17T11:43:32.205+0000 7fa09f8628c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:36.600 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:43:36.600 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:43:36.600 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:43:36.600 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:43:36.600 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:43:36.600 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:36.600 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:43:36.601 INFO:tasks.workunit.client.0.trial128.stderr:164704: 0 2026-02-17T11:43:36.601 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:36.601 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:36.601 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:36.601 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:36.601 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:36.601 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:43:36.601 INFO:tasks.workunit.client.0.trial128.stderr:164704: 1 2026-02-17T11:43:36.601 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:36.601 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:36.601 INFO:tasks.workunit.client.0.trial128.stderr:164704: 2026-02-17T11:43:33.513+0000 7fa09f8628c0 -1 Falling back to public interface 2026-02-17T11:43:36.601 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:36.602 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:36.602 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:36.602 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:43:36.602 INFO:tasks.workunit.client.0.trial128.stderr:164704: 2 2026-02-17T11:43:36.602 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:36.602 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:36.602 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:36.602 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:36.602 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:36.602 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:43:36.602 INFO:tasks.workunit.client.0.trial128.stderr:164704: 3 2026-02-17T11:43:36.602 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:36.603 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:36.603 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:36.807 INFO:tasks.workunit.client.0.trial128.stderr:164704: 2026-02-17T11:43:36.589+0000 7fa09f8628c0 164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T11:43:36.807 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T11:43:36.807 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:43:36.807 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:43:36.807 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:43:36.807 INFO:tasks.workunit.client.0.trial128.stderr:164701: 2026-02-17T11:43:32.281+0000 7f288da478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:36.808 INFO:tasks.workunit.client.0.trial128.stderr:164701: 2026-02-17T11:43:32.281+0000 7f288da478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:36.808 INFO:tasks.workunit.client.0.trial128.stderr:164701: 2026-02-17T11:43:32.281+0000 7f288da478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:36.808 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T11:43:36.808 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:43:36.808 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:43:36.808 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:43:36.808 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:43:36.808 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:36.808 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:43:36.808 INFO:tasks.workunit.client.0.trial128.stderr:164701: 0 2026-02-17T11:43:36.808 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:36.808 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:43:36.809 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:36.809 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:36.809 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:36.809 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:43:36.809 INFO:tasks.workunit.client.0.trial128.stderr:164701: 1 2026-02-17T11:43:36.809 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:36.809 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:43:36.809 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:36.809 INFO:tasks.workunit.client.0.trial128.stderr:164701: 2026-02-17T11:43:33.797+0000 7f288da478c0 -1 Falling back to public interface 2026-02-17T11:43:36.809 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:36.809 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:36.809 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:43:36.810 INFO:tasks.workunit.client.0.trial128.stderr:164701: 2 2026-02-17T11:43:36.810 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:36.810 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:43:36.810 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:36.810 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:36.810 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:36.810 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:43:36.810 INFO:tasks.workunit.client.0.trial128.stderr:164701: 3 2026-02-17T11:43:36.810 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:36.810 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:43:36.810 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:37.468 INFO:tasks.workunit.client.0.trial128.stderr:164701: 2026-02-17T11:43:36.797+0000 7f288da478c0 -1 osd.1 73 log_to_monitors true 2026-02-17T11:43:37.468 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:37.468 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:37.468 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:43:37.469 INFO:tasks.workunit.client.0.trial128.stderr:164704: 4 2026-02-17T11:43:37.469 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:37.469 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:37.469 INFO:tasks.workunit.client.0.trial128.stderr:164704: osd.1 up in weight 1 up_from 77 up_thru 77 down_at 74 last_clean_interval [72,73) [v2:127.0.0.1:6818/126965442,v1:127.0.0.1:6819/126965442] [v2:127.0.0.1:6820/126965442,v1:127.0.0.1:6821/126965442] exists,up 0107cbfc-eed6-42f4-b707-bfb3f3155b29 2026-02-17T11:43:37.469 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:43:37.469 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:43:37.469 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:43:37.469 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:43:37.469 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:43:37.469 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:43:37.470 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:43:37.470 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:43:37.470 INFO:tasks.workunit.client.0.trial128.stderr:164704: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:43:37.470 INFO:tasks.workunit.client.0.trial128.stderr:164704: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:43:37.470 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:43:37.470 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:43:37.470 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:43:37.470 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:43:37.470 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:43:37.470 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:43:37.470 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:43:37.470 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:43:37.470 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:43:37.471 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:43:37.471 INFO:tasks.workunit.client.0.trial128.stderr:164704: 1 2026-02-17T11:43:37.471 INFO:tasks.workunit.client.0.trial128.stderr:164704: 2 2026-02-17T11:43:37.471 INFO:tasks.workunit.client.0.trial128.stderr:164704: 3' 2026-02-17T11:43:37.471 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:43:37.471 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:37.471 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:43:37.471 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836516 2026-02-17T11:43:37.471 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836516 2026-02-17T11:43:37.471 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836516' 2026-02-17T11:43:37.471 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:37.520 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers-1 osd.3 73 log_to_monitors true 2026-02-17T11:43:37.520 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:37.520 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:37.520 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:43:37.521 INFO:tasks.workunit.client.0.trial128.stderr:164701: 4 2026-02-17T11:43:37.521 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:37.521 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:43:37.521 INFO:tasks.workunit.client.0.trial128.stderr:164701: osd.3 up in weight 1 up_from 79 up_thru 79 down_at 74 last_clean_interval [57,73) [v2:127.0.0.1:6826/1687839543,v1:127.0.0.1:6827/1687839543] [v2:127.0.0.1:6828/1687839543,v1:127.0.0.1:6829/1687839543] exists,up bdb03fba-ae78-43d3-8520-91782827db02 2026-02-17T11:43:37.521 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:43:37.521 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:43:37.521 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:43:37.521 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:43:37.521 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:43:37.521 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:43:37.521 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:43:37.522 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:43:37.522 INFO:tasks.workunit.client.0.trial128.stderr:164701: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:43:37.522 INFO:tasks.workunit.client.0.trial128.stderr:164701: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:43:37.522 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:43:37.522 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:43:37.522 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:43:37.522 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:43:37.522 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:43:37.522 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:43:37.522 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:43:37.522 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:43:37.522 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:43:37.523 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:43:37.523 INFO:tasks.workunit.client.0.trial128.stderr:164701: 1 2026-02-17T11:43:37.523 INFO:tasks.workunit.client.0.trial128.stderr:164701: 2 2026-02-17T11:43:37.523 INFO:tasks.workunit.client.0.trial128.stderr:164701: 3' 2026-02-17T11:43:37.523 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:43:37.523 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:37.523 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:43:37.523 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836517 2026-02-17T11:43:37.523 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836517 2026-02-17T11:43:37.523 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517' 2026-02-17T11:43:37.523 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:39.008 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-hel.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:43:39.009 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=330712481794 2026-02-17T11:43:39.009 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 330712481794 2026-02-17T11:43:39.009 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836516 1-330712481794' 2026-02-17T11:43:39.009 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:39.009 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:43:39.009 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678022 2026-02-17T11:43:39.009 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678022 2026-02-17T11:43:39.010 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836516 1-330712481794 2-304942678022' 2026-02-17T11:43:39.010 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:39.010 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:43:39.010 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=339302416386 2026-02-17T11:43:39.010 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 339302416386 2026-02-17T11:43:39.010 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836516 1-330712481794 2-304942678022 3-339302416386' 2026-02-17T11:43:39.010 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:39.010 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836516 2026-02-17T11:43:39.010 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:39.010 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:43:39.011 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836516 2026-02-17T11:43:39.011 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:39.011 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836516 2026-02-17T11:43:39.011 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836516' 2026-02-17T11:43:39.011 INFO:tasks.workunit.client.0.trial128.stderr:164704: waiting osd.0 seq 21474836516 2026-02-17T11:43:39.011 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:43:39.011 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836515 -lt 21474836516 2026-02-17T11:43:39.011 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:43:39.011 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:43:39.011 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:43:39.012 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836517 -lt 21474836516 2026-02-17T11:43:39.012 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:39.012 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-330712481794 2026-02-17T11:43:39.012 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:39.012 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:43:39.012 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-330712481794 2026-02-17T11:43:39.012 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:39.053 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:43:39.054 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=330712481795 2026-02-17T11:43:39.054 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 330712481795 2026-02-17T11:43:39.054 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-330712481795' 2026-02-17T11:43:39.054 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:39.054 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:43:39.054 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678023 2026-02-17T11:43:39.054 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678023 2026-02-17T11:43:39.054 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-330712481795 2-304942678023' 2026-02-17T11:43:39.054 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:39.054 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:43:39.054 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=339302416387 2026-02-17T11:43:39.055 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 339302416387 2026-02-17T11:43:39.055 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836517 1-330712481795 2-304942678023 3-339302416387' 2026-02-17T11:43:39.055 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:39.055 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836517 2026-02-17T11:43:39.055 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:39.055 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:43:39.055 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836517 2026-02-17T11:43:39.055 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:39.055 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836517 2026-02-17T11:43:39.055 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836517' 2026-02-17T11:43:39.055 INFO:tasks.workunit.client.0.trial128.stderr:164701: waiting osd.0 seq 21474836517 2026-02-17T11:43:39.055 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:43:39.056 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836515 -lt 21474836517 2026-02-17T11:43:39.056 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:43:39.056 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:43:39.056 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:43:39.056 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836517 -lt 21474836517 2026-02-17T11:43:39.056 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:39.056 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-330712481795 2026-02-17T11:43:39.056 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:39.056 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:43:39.056 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-330712481795 2026-02-17T11:43:39.056 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:39.684 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/uu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=330712481794 2026-02-17T11:43:39.684 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 330712481794' 2026-02-17T11:43:39.684 INFO:tasks.workunit.client.0.trial128.stderr:164704: waiting osd.1 seq 330712481794 2026-02-17T11:43:39.684 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:43:39.684 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 330712481795 -lt 330712481794 2026-02-17T11:43:39.684 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:39.684 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-304942678022 2026-02-17T11:43:39.685 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:39.685 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:43:39.685 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-304942678022 2026-02-17T11:43:39.685 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:39.685 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678022 2026-02-17T11:43:39.685 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 304942678022' 2026-02-17T11:43:39.685 INFO:tasks.workunit.client.0.trial128.stderr:164704: waiting osd.2 seq 304942678022 2026-02-17T11:43:39.685 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:43:39.685 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678023 -lt 304942678022 2026-02-17T11:43:39.685 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:39.686 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-339302416386 2026-02-17T11:43:39.686 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:39.686 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:43:39.686 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-339302416386 2026-02-17T11:43:39.686 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:39.686 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=339302416386 2026-02-17T11:43:39.686 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 339302416386' 2026-02-17T11:43:39.686 INFO:tasks.workunit.client.0.trial128.stderr:164704: waiting osd.3 seq 339302416386 2026-02-17T11:43:39.686 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:43:39.686 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 339302416387 -lt 339302416386 2026-02-17T11:43:39.686 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:43:39.686 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:39.687 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:39.687 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:43:39.687 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:39.687 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:39.687 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:39.687 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:39.689 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stalbuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=330712481795 2026-02-17T11:43:39.689 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 330712481795' 2026-02-17T11:43:39.689 INFO:tasks.workunit.client.0.trial128.stderr:164701: waiting osd.1 seq 330712481795 2026-02-17T11:43:39.690 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:43:39.690 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 330712481795 -lt 330712481795 2026-02-17T11:43:39.690 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:39.690 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-304942678023 2026-02-17T11:43:39.690 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:39.690 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:43:39.690 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-304942678023 2026-02-17T11:43:39.690 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:39.690 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678023 2026-02-17T11:43:39.690 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 304942678023' 2026-02-17T11:43:39.691 INFO:tasks.workunit.client.0.trial128.stderr:164701: waiting osd.2 seq 304942678023 2026-02-17T11:43:39.691 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:43:39.691 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678023 -lt 304942678023 2026-02-17T11:43:39.691 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:39.691 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-339302416387 2026-02-17T11:43:39.691 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:39.691 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:43:39.691 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-339302416387 2026-02-17T11:43:39.691 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:39.691 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=339302416387 2026-02-17T11:43:39.691 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 339302416387' 2026-02-17T11:43:39.692 INFO:tasks.workunit.client.0.trial128.stderr:164701: waiting osd.3 seq 339302416387 2026-02-17T11:43:39.692 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:43:39.692 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 339302416387 -lt 339302416387 2026-02-17T11:43:39.692 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:43:39.692 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:39.692 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:39.692 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:43:39.692 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:39.692 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:39.692 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:39.693 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:40.024 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:40.024 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:40.024 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:40.025 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:43:40.025 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:40.025 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:40.025 INFO:tasks.workunit.client.0.trial128.stderr:164701: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:40.025 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:43:40.025 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:43:40.025 INFO:tasks.workunit.client.0.trial128.stderr:164701: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:43:40.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:43:40.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:43:40.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 164700 2026-02-17T11:43:40.029 INFO:tasks.workunit.client.0.trial128.stderr:e") | not)' 2026-02-17T11:43:40.030 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:40.030 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:40.030 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:43:40.030 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:40.030 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:40.030 INFO:tasks.workunit.client.0.trial128.stderr:164704: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:40.030 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:43:40.030 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:43:40.030 INFO:tasks.workunit.client.0.trial128.stderr:164704: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:43:40.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:43:40.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T11:43:40.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T11:43:40.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T11:43:40.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: corrupt_and_repair_two: return_code=0 2026-02-17T11:43:40.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:198: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T11:43:40.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-17T11:43:40.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:43:40.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:43:40.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:43:40.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:43:40.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: local pg=2.0 2026-02-17T11:43:40.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:204: corrupt_and_repair_two: repair 2.0 2026-02-17T11:43:40.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T11:43:40.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T11:43:40.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:43:40.183 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:43:40.183 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:43:40.183 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:43:40.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:43:17.336523+0000 2026-02-17T11:43:40.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T11:43:40.480 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T11:43:40.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T11:43:17.336523+0000 2026-02-17T11:43:40.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:43:40.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:43:17.336523+0000 2026-02-17T11:43:40.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:43:40.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:43:40.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:43:40.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:43:40.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:43:40.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:43:40.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:43:40.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:43:40.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:43:17.336523+0000 '>' 2026-02-17T11:43:17.336523+0000 2026-02-17T11:43:40.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:43:41.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:43:41.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:43:41.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:43:41.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:43:41.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:43:41.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:43:41.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:43:41.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:43:17.336523+0000 '>' 2026-02-17T11:43:17.336523+0000 2026-02-17T11:43:41.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:43:42.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:43:42.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:43:42.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:43:42.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:43:42.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:43:42.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:43:42.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:43:42.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:43:40.787003+0000 '>' 2026-02-17T11:43:17.336523+0000 2026-02-17T11:43:42.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:43:42.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: corrupt_and_repair_two: pids= 2026-02-17T11:43:42.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:43:42.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:43:42.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:43:42.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 167604"' 2026-02-17T11:43:42.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 167604' 2026-02-17T11:43:42.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:43:42.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:43:42.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:43:42.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 167606"' 2026-02-17T11:43:42.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:43:42.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 167606' 2026-02-17T11:43:42.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:211: corrupt_and_repair_two: wait_background pids 2026-02-17T11:43:42.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 167604 167606' 2026-02-17T11:43:42.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T11:43:42.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:43:42.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 167604 2026-02-17T11:43:42.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/167607: /' 2026-02-17T11:43:42.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:43:42.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/167609: /' 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:43:45.113 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:43:45.114 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:43:45.114 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T11:43:45.114 INFO:tasks.workunit.client.0.trial128.stderr:167609: _ 2026-02-17T11:43:45.114 INFO:tasks.workunit.client.0.trial128.stderr:167609: hinfo_key 2026-02-17T11:43:45.114 INFO:tasks.workunit.client.0.trial128.stderr:167609: snapset 2026-02-17T11:43:45.114 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:43:45.114 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:43:45.114 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:43:45.114 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:43:45.114 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:43:45.114 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:43:45.114 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:43:45.114 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:43:45.117 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:43:45.117 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:43:45.117 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:43:45.117 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:43:45.117 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:43:45.117 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:43:45.117 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:43:45.117 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:43:45.117 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:43:45.117 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:43:45.117 INFO:tasks.workunit.client.0.trial128.stderr:167609: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:43:45.117 INFO:tasks.workunit.client.0.trial128.stderr:167609: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:43:45.117 INFO:tasks.workunit.client.0.trial128.stderr:167609: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:43:45.117 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:45.118 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:45.118 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:43:45.118 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:43:45.118 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:43:45.118 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:43:45.118 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:43:45.118 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:43:45.118 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:43:45.118 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:43:45.118 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:43:45.118 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:43:45.118 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:43:45.118 INFO:tasks.workunit.client.0.trial128.stderr:167609: start osd.1 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_rec167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: _ 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: hinfo_key 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: snapset 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T11:43:45.120 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:43:45.121 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:43:45.121 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T11:43:45.121 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:43:45.121 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:43:45.121 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:43:45.123 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:43:45.123 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:43:45.123 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:43:45.123 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:43:45.123 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:43:45.123 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:43:45.123 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:43:45.123 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:43:45.123 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:43:45.123 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:43:45.123 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:43:45.123 INFO:tasks.workunit.client.0.trial128.stderr:167607: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:43:45.123 INFO:tasks.workunit.client.0.trial128.stderr:167607: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:43:45.123 INFO:tasks.workunit.client.0.trial128.stderr:167607: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:43:45.123 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:45.124 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:45.124 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:43:45.124 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:43:45.124 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:43:45.124 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:43:45.124 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:43:45.124 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:43:45.124 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:43:45.124 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:43:45.124 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:43:45.124 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:43:45.124 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T11:43:45.124 INFO:tasks.workunit.client.0.trial128.stderr:167607: start osd.3 2026-02-17T11:43:48.912 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:43:48.912 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T11:43:48.912 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T11:43:48.912 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:43:48.912 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:43:48.912 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:43:48.913 INFO:tasks.workunit.client.0.trial128.stderr:167607: 2026-02-17T11:43:45.128+0000 7f4f55cd38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:48.913 INFO:tasks.workunit.client.0.trial128.stderr:167607: 2026-02-17T11:43:45.132+0000 7f4f55cd38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:48.913 INFO:tasks.workunit.client.0.trial128.stderr:167607: 2026-02-17T11:43:45.132+0000 7f4f55cd38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:48.913 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T11:43:48.913 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:43:48.913 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:43:48.913 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:43:48.913 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:43:48.913 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:48.913 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:43:48.913 INFO:tasks.workunit.client.0.trial128.stderr:167607: 0 2026-02-17T11:43:48.913 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:48.914 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:43:48.914 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:48.914 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:48.914 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:48.914 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:43:48.914 INFO:tasks.workunit.client.0.trial128.stderr:167607: 1 2026-02-17T11:43:48.914 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:48.914 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:43:48.914 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:48.914 INFO:tasks.workunit.client.0.trial128.stderr:167607: 2026-02-17T11:43:46.620+0000 7f4f55cd38c0 -1 Falling back to public interface 2026-02-17T11:43:48.914 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:48.914 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:48.915 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:43:48.915 INFO:tasks.workunit.client.0.trial128.stderr:167607: 2 2026-02-17T11:43:48.915 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:48.915 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:43:48.915 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:48.915 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:48.915 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:48.915 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:43:48.915 INFO:tasks.workunit.client.0.trial128.stderr:167607: 3 2026-02-17T11:43:48.915 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:48.915 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:43:49.417 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.clientovery_ops 2026-02-17T11:43:49.418 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:43:49.418 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:43:49.418 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:43:49.418 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:43:49.418 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:43:49.418 INFO:tasks.workunit.client.0.trial128.stderr:167609: 2026-02-17T11:43:45.124+0000 7f2d65f6e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:49.418 INFO:tasks.workunit.client.0.trial128.stderr:167609: 2026-02-17T11:43:45.124+0000 7f2d65f6e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:49.418 INFO:tasks.workunit.client.0.trial128.stderr:167609: 2026-02-17T11:43:45.124+0000 7f2d65f6e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:49.418 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:43:49.418 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:43:49.418 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:43:49.419 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:43:49.419 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:43:49.419 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:49.419 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:43:49.419 INFO:tasks.workunit.client.0.trial128.stderr:167609: 0 2026-02-17T11:43:49.419 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:49.419 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:49.419 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:49.419 INFO:tasks.workunit.client.0.trial128.stderr:167609: 2026-02-17T11:43:46.396+0000 7f2d65f6e8c0 -1 Falling back to public interface 2026-02-17T11:43:49.419 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:49.419 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:49.419 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:43:49.419 INFO:tasks.workunit.client.0.trial128.stderr:167609: 1 2026-02-17T11:43:49.420 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:49.420 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:49.420 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:49.420 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:49.420 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:49.420 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:43:49.420 INFO:tasks.workunit.client.0.trial128.stderr:167609: 2 2026-02-17T11:43:49.420 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:49.420 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:49.420 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:49.420 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:49.420 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:49.421 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:43:49.421 INFO:tasks.workunit.client.0.trial128.stderr:167609: 3 2026-02-17T11:43:49.421 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:49.421 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:49.421 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:50.379 INFO:tasks.workunit.client.0.trial128.stderr:167609: 2026-02-17T11:43:49.408+0000 7f2d65f6e8c0 -1 osd.1 81 log_to_monitors true 2026-02-17T11:43:50.379 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:50.380 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:50.380 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:43:50.380 INFO:tasks.workunit.client.0.trial128.stderr:167609: 4 2026-02-17T11:43:50.380 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:50.380 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:43:50.380 INFO:tasks.workunit.client.0.trial128.stderr:167609: osd.1 up in weight 1 up_from 85 up_thru 85 down_at 82 last_clean_interval [77,81) [v2:127.0.0.1:6818/66916284,v1:127.0.0.1:6819/66916284] [v2:127.0.0.1:6820/66916284,v1:127.0.0.1:6821/66916284] exists,up 0107cbfc-eed6-42f4-b707-bfb3f3155b29 2026-02-17T11:43:50.380 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:43:50.380 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:43:50.380 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:43:50.380 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:43:50.380 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:43:50.380 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:43:50.381 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:43:50.381 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:43:50.381 INFO:tasks.workunit.client.0.trial128.stderr:167609: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:43:50.381 INFO:tasks.workunit.client.0.trial128.stderr:167609: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:43:50.381 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:43:50.381 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:43:50.381 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:43:50.381 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:43:50.381 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:43:50.381 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:43:50.381 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:43:50.381 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:43:50.381 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:43:50.382 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:43:50.382 INFO:tasks.workunit.client.0.trial128.stderr:167609: 1 2026-02-17T11:43:50.382 INFO:tasks.workunit.client.0.trial128.stderr:167609: 2 2026-02-17T11:43:50.382 INFO:tasks.workunit.client.0.trial128.stderr:167609: 3' 2026-02-17T11:43:50.382 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:43:50.382 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:50.382 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:43:50.382 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836521 2026-02-17T11:43:50.382 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836521 2026-02-17T11:43:50.382 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521' 2026-02-17T11:43:50.382 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-h.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167607: 2026-02-17T11:43:49.604+0000 7f4f55cd38c0 -1 osd.3 81 log_to_monitors true 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167607: 4 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167607: osd.3 up in weight 1 up_from 87 up_thru 87 down_at 82 last_clean_interval [79,81) [v2:127.0.0.1:6826/204940841,v1:127.0.0.1:6827/204940841] [v2:127.0.0.1:6828/204940841,v1:127.0.0.1:6829/204940841] exists,up bdb03fba-ae78-43d3-8520-91782827db02 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:43:50.402 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: 1 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: 2 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: 3' 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:43:50.403 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836522 2026-02-17T11:43:50.404 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836522 2026-02-17T11:43:50.404 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522' 2026-02-17T11:43:53.064 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:elpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:43:53.065 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=365072220162 2026-02-17T11:43:53.065 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 365072220162 2026-02-17T11:43:53.065 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521 1-365072220162' 2026-02-17T11:43:53.065 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:53.065 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:43:53.065 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678027 2026-02-17T11:43:53.065 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678027 2026-02-17T11:43:53.065 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521 1-365072220162 2-304942678027' 2026-02-17T11:43:53.065 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:53.065 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:43:53.066 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=373662154754 2026-02-17T11:43:53.066 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 373662154754 2026-02-17T11:43:53.066 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836521 1-365072220162 2-304942678027 3-373662154754' 2026-02-17T11:43:53.066 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:53.066 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836521 2026-02-17T11:43:53.066 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:53.066 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:43:53.066 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836521 2026-02-17T11:43:53.066 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:53.066 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836521 2026-02-17T11:43:53.066 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836521' 2026-02-17T11:43:53.066 INFO:tasks.workunit.client.0.trial128.stderr:167609: waiting osd.0 seq 21474836521 2026-02-17T11:43:53.066 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:43:53.067 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836520 -lt 21474836521 2026-02-17T11:43:53.067 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:43:53.067 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:43:53.067 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:43:53.067 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836520 -lt 21474836521 2026-02-17T11:43:53.067 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:43:53.067 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:43:53.067 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:43:53.067 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836522 -lt 21474836521 2026-02-17T11:43:53.067 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:53.092 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:53.092 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:43:53.093 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=365072220163 2026-02-17T11:43:53.093 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 365072220163 2026-02-17T11:43:53.093 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-365072220163' 2026-02-17T11:43:53.093 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:53.093 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:43:53.093 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678028 2026-02-17T11:43:53.093 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678028 2026-02-17T11:43:53.093 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-365072220163 2-304942678028' 2026-02-17T11:43:53.093 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:43:53.093 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:43:53.093 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=373662154755 2026-02-17T11:43:53.094 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 373662154755 2026-02-17T11:43:53.094 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836522 1-365072220163 2-304942678028 3-373662154755' 2026-02-17T11:43:53.094 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:53.094 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836522 2026-02-17T11:43:53.094 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:53.094 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:43:53.094 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836522 2026-02-17T11:43:53.094 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:53.094 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836522 2026-02-17T11:43:53.094 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836522' 2026-02-17T11:43:53.094 INFO:tasks.workunit.client.0.trial128.stderr:167607: waiting osd.0 seq 21474836522 2026-02-17T11:43:53.094 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:43:53.095 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836520 -lt 21474836522 2026-02-17T11:43:53.095 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:43:53.095 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:43:53.095 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:43:53.095 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836520 -lt 21474836522 2026-02-17T11:43:53.095 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:43:53.095 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:43:53.095 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:43:53.095 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836522 -lt 21474836522 2026-02-17T11:43:53.750 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:53.750 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-365072220163 2026-02-17T11:43:53.750 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:53.751 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:43:53.751 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-365072220163 2026-02-17T11:43:53.751 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:53.751 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=365072220163 2026-02-17T11:43:53.751 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 365072220163' 2026-02-17T11:43:53.751 INFO:tasks.workunit.client.0.trial128.stderr:167607: waiting osd.1 seq 365072220163 2026-02-17T11:43:53.751 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:43:53.751 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 365072220163 -lt 365072220163 2026-02-17T11:43:53.751 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:53.751 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-304942678028 2026-02-17T11:43:53.751 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:53.751 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:43:53.752 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-304942678028 2026-02-17T11:43:53.752 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:53.752 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678028 2026-02-17T11:43:53.752 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 304942678028' 2026-02-17T11:43:53.752 INFO:tasks.workunit.client.0.trial128.stderr:167607: waiting osd.2 seq 304942678028 2026-02-17T11:43:53.752 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:43:53.752 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678028 -lt 304942678028 2026-02-17T11:43:53.752 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:53.752 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-373662154755 2026-02-17T11:43:53.752 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:53.753 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:43:53.753 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-373662154755 2026-02-17T11:43:53.753 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:53.753 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=373662154755 2026-02-17T11:43:53.753 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 373662154755' 2026-02-17T11:43:53.753 INFO:tasks.workunit.client.0.trial128.stderr:167607: waiting osd.3 seq 373662154755 2026-02-17T11:43:53.753 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:43:53.753 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 373662154755 -lt 373662154755 2026-02-17T11:43:53.753 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:43:53.753 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:53.753 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:53.753 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:43:53.757 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-hel1-365072220162 2026-02-17T11:43:53.757 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:53.757 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:43:53.757 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-365072220162 2026-02-17T11:43:53.758 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:53.758 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=365072220162 2026-02-17T11:43:53.758 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 365072220162' 2026-02-17T11:43:53.758 INFO:tasks.workunit.client.0.trial128.stderr:167609: waiting osd.1 seq 365072220162 2026-02-17T11:43:53.758 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:43:53.758 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 365072220163 -lt 365072220162 2026-02-17T11:43:53.758 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:53.758 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-304942678027 2026-02-17T11:43:53.758 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:53.758 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:43:53.758 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-304942678027 2026-02-17T11:43:53.758 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:53.758 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678027 2026-02-17T11:43:53.759 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 304942678027' 2026-02-17T11:43:53.759 INFO:tasks.workunit.client.0.trial128.stderr:167609: waiting osd.2 seq 304942678027 2026-02-17T11:43:53.759 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:43:53.759 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678028 -lt 304942678027 2026-02-17T11:43:53.759 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:43:53.759 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-373662154754 2026-02-17T11:43:53.759 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:43:53.759 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:43:53.759 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-373662154754 2026-02-17T11:43:53.759 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:43:53.759 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=373662154754 2026-02-17T11:43:53.759 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 373662154754' 2026-02-17T11:43:53.760 INFO:tasks.workunit.client.0.trial128.stderr:167609: waiting osd.3 seq 373662154754 2026-02-17T11:43:53.760 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:43:53.760 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 373662154755 -lt 373662154754 2026-02-17T11:43:53.760 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:43:53.760 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:53.760 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:53.760 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:43:53.760 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:43:54.100 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:54.100 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:54.100 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:54.100 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:54.100 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:54.100 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:54.101 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:43:54.101 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:54.101 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:54.101 INFO:tasks.workunit.client.0.trial128.stderr:167609: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:54.101 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:43:54.101 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:43:54.101 INFO:tasks.workunit.client.0.trial128.stderr:167609: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:43:54.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:43:54.103 INFO:tasks.workunit.client.0.trial128.stderr:pers.sh:1667: wait_for_clean: true 2026-02-17T11:43:54.103 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:43:54.103 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:43:54.103 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:43:54.103 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:43:54.103 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:43:54.103 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:43:54.103 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:43:54.103 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:43:54.104 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:43:54.104 INFO:tasks.workunit.client.0.trial128.stderr:167607: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:43:54.104 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:43:54.104 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:43:54.104 INFO:tasks.workunit.client.0.trial128.stderr:167607: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:43:54.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:43:54.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:43:54.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 167606 2026-02-17T11:43:54.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T11:43:54.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T11:43:54.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T11:43:54.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:212: corrupt_and_repair_two: return_code=0 2026-02-17T11:43:54.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:213: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T11:43:54.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:43:54.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:43:54.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T11:43:54.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:43:54.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:43:54.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:43:54.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:43:54.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:43:54.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:43:54.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:43:54.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:43:54.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:43:54.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:43:54.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:43:54.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:43:54.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:43:54.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:43:54.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:43:54.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:43:54.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:43:54.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:43:54.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:43:54.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:43:54.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:43:54.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:43:54.270 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:43:54.270 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:43:54.270 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:43:54.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:43:54.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:43:54.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:43:54.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T11:43:54.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T11:43:54.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T11:43:54.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T11:43:54.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:43:54.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:43:54.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:43:54.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:43:54.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:43:54.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:43:54.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:43:54.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:43:54.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:43:54.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:43:54.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:43:54.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:43:54.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:43:54.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:43:54.279 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:43:54.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:43:54.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:43:54.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:43:54.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:43:54.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:43:54.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:43:54.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:43:54.285 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:43:54.285 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:43:54.285 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:43:54.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:43:54.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:43:54.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:43:54.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T11:43:54.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:43:54.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:43:54.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:43:54.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T11:43:54.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:43:54.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T11:43:54.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:43:54.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T11:43:54.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_and_repair_lrc_appends td/osd-scrub-repair 2026-02-17T11:43:54.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:842: TEST_corrupt_and_repair_lrc_appends: corrupt_and_repair_lrc td/osd-scrub-repair false 2026-02-17T11:43:54.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:825: corrupt_and_repair_lrc: local dir=td/osd-scrub-repair 2026-02-17T11:43:54.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:826: corrupt_and_repair_lrc: local allow_overwrites=false 2026-02-17T11:43:54.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:827: corrupt_and_repair_lrc: local poolname=ecpool 2026-02-17T11:43:54.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:829: corrupt_and_repair_lrc: run_mon td/osd-scrub-repair a 2026-02-17T11:43:54.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T11:43:54.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T11:43:54.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T11:43:54.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T11:43:54.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T11:43:54.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T11:43:54.312 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T11:43:54.312 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:43:54.312 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:43:54.312 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:43:54.312 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:43:54.312 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:43:54.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:54.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T11:43:54.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T11:43:54.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T11:43:54.342 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:43:54.342 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:43:54.342 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T11:43:54.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T11:43:54.343 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:43:54.343 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:43:54.343 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:43:54.343 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:43:54.343 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:43:54.343 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:43:54.344 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:43:54.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:43:54.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T11:43:54.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T11:43:54.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:43:54.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:43:54.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T11:43:54.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T11:43:54.403 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:43:54.403 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:43:54.403 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:43:54.403 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:43:54.403 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:43:54.403 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:43:54.403 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:43:54.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:43:54.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T11:43:54.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:830: corrupt_and_repair_lrc: run_mgr td/osd-scrub-repair x 2026-02-17T11:43:54.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T11:43:54.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T11:43:54.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T11:43:54.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T11:43:54.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T11:43:54.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T11:43:54.553 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T11:43:54.553 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:43:54.553 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:43:54.554 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:43:54.554 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:43:54.554 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:43:54.554 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:54.555 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:43:54.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:43:54.573 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: seq 0 9 2026-02-17T11:43:54.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:43:54.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 0 2026-02-17T11:43:54.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:43:54.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:43:54.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T11:43:54.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:43:54.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:43:54.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:43:54.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:43:54.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:43:54.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:43:54.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:43:54.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:43:54.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:43:54.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:43:54.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:43:54.583 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:43:54.584 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:43:54.584 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:43:54.584 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:43:54.584 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:43:54.584 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:43:54.584 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:54.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:43:54.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:43:54.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:43:54.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:43:54.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:43:54.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:43:54.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:43:54.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:43:54.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:43:54.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:43:54.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:43:54.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:43:54.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:43:54.587 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:43:54.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a81cdfbe-c7ef-41fe-a962-ee92e07ac77c 2026-02-17T11:43:54.588 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 a81cdfbe-c7ef-41fe-a962-ee92e07ac77c 2026-02-17T11:43:54.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 a81cdfbe-c7ef-41fe-a962-ee92e07ac77c' 2026-02-17T11:43:54.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:43:54.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB6VJRpYvWjIhAAUiA7QCmNScBTgwsjlVDOwA== 2026-02-17T11:43:54.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB6VJRpYvWjIhAAUiA7QCmNScBTgwsjlVDOwA=="}' 2026-02-17T11:43:54.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a81cdfbe-c7ef-41fe-a962-ee92e07ac77c -i td/osd-scrub-repair/0/new.json 2026-02-17T11:43:54.677 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:43:54.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T11:43:54.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB6VJRpYvWjIhAAUiA7QCmNScBTgwsjlVDOwA== --osd-uuid a81cdfbe-c7ef-41fe-a962-ee92e07ac77c 2026-02-17T11:43:54.701 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:43:54.692+0000 7f5e0598b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:54.703 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:43:54.692+0000 7f5e0598b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:54.704 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:43:54.692+0000 7f5e0598b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:54.704 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:43:54.696+0000 7f5e0598b8c0 -1 bdev(0x560e58f3dc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:43:54.704 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:43:54.696+0000 7f5e0598b8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T11:43:57.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T11:43:57.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:43:57.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T11:43:57.522 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:43:57.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:43:57.710 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:43:57.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T11:43:57.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:43:57.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:43:57.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:43:57.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:43:57.734 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:43:57.724+0000 7f3aab97d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:57.735 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:43:57.724+0000 7f3aab97d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:57.736 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:43:57.728+0000 7f3aab97d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:43:57.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T11:43:57.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:43:57.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:43:57.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:43:57.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:43:57.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:57.864 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:43:57.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:43:57.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:57.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:43:58.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:59.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:43:59.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:43:59.019 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:43:59.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:43:59.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:43:59.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:43:59.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:43:59.270 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:43:59.260+0000 7f3aab97d8c0 -1 Falling back to public interface 2026-02-17T11:44:00.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:00.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:00.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:44:00.172 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:44:00.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:00.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:44:00.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:01.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:01.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:01.335 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:44:01.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:44:01.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:01.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:44:01.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:02.236 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:02.224+0000 7f3aab97d8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:44:02.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:02.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:02.491 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:44:02.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:44:02.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:02.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:44:02.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:03.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:03.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:03.646 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:44:03.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:44:03.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:03.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:44:03.797 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/113864492,v1:127.0.0.1:6803/113864492] [v2:127.0.0.1:6804/113864492,v1:127.0.0.1:6805/113864492] exists,up a81cdfbe-c7ef-41fe-a962-ee92e07ac77c 2026-02-17T11:44:03.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:44:03.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:44:03.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:44:03.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:44:03.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 1 2026-02-17T11:44:03.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:44:03.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:44:03.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T11:44:03.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:44:03.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:44:03.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:44:03.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:44:03.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:44:03.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:44:03.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:44:03.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:44:03.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:44:03.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:44:03.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:44:03.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:44:03.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:44:03.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:44:03.803 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:44:03.803 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:44:03.803 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:44:03.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:44:03.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:44:03.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:44:03.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:44:03.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:44:03.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:44:03.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:44:03.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:44:03.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:44:03.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:44:03.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:44:03.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:44:03.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:44:03.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:44:03.805 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:44:03.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=37d97226-4b96-4826-bc43-85c9e873a112 2026-02-17T11:44:03.805 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 37d97226-4b96-4826-bc43-85c9e873a112 2026-02-17T11:44:03.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 37d97226-4b96-4826-bc43-85c9e873a112' 2026-02-17T11:44:03.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:44:03.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCDVJRpBa8iMBAAeRR30GRl3cCx3NUIdTDkEA== 2026-02-17T11:44:03.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCDVJRpBa8iMBAAeRR30GRl3cCx3NUIdTDkEA=="}' 2026-02-17T11:44:03.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 37d97226-4b96-4826-bc43-85c9e873a112 -i td/osd-scrub-repair/1/new.json 2026-02-17T11:44:03.971 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:44:03.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T11:44:03.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCDVJRpBa8iMBAAeRR30GRl3cCx3NUIdTDkEA== --osd-uuid 37d97226-4b96-4826-bc43-85c9e873a112 2026-02-17T11:44:04.006 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:03.996+0000 7f47e5cd98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:04.007 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:03.996+0000 7f47e5cd98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:04.008 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:04.000+0000 7f47e5cd98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:04.009 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:04.000+0000 7f47e5cd98c0 -1 bdev(0x56260a6cbc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:44:04.009 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:04.000+0000 7f47e5cd98c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T11:44:06.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T11:44:06.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:44:06.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T11:44:06.263 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:44:06.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:44:06.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T11:44:06.463 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:44:06.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:44:06.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:44:06.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:44:06.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:44:06.485 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:06.476+0000 7f1f0c6b98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:06.486 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:06.476+0000 7f1f0c6b98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:06.488 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:06.476+0000 7f1f0c6b98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:06.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T11:44:06.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:44:06.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:44:06.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:44:06.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:44:06.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:06.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:44:06.612 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:44:06.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:06.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:44:06.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:07.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:07.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:07.769 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:44:07.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:44:07.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:07.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:44:07.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:07.982 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:07.972+0000 7f1f0c6b98c0 -1 Falling back to public interface 2026-02-17T11:44:08.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:08.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:08.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:44:08.928 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:44:08.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:08.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:44:09.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:10.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:10.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:10.087 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:44:10.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:44:10.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:10.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:44:10.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:10.973 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:10.964+0000 7f1f0c6b98c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:44:11.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:11.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:11.242 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:44:11.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:44:11.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:11.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:44:11.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:12.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:12.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:12.401 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:44:12.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:44:12.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:12.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:44:12.555 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1999221754,v1:127.0.0.1:6811/1999221754] [v2:127.0.0.1:6812/1999221754,v1:127.0.0.1:6813/1999221754] exists,up 37d97226-4b96-4826-bc43-85c9e873a112 2026-02-17T11:44:12.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:44:12.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:44:12.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:44:12.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:44:12.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 2 2026-02-17T11:44:12.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:44:12.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:44:12.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T11:44:12.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:44:12.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:44:12.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:44:12.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:44:12.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:44:12.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:44:12.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:44:12.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:44:12.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:44:12.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:44:12.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:44:12.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:44:12.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:44:12.562 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:44:12.562 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:44:12.562 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:44:12.562 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:44:12.562 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:44:12.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:44:12.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:44:12.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:44:12.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:44:12.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:44:12.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:44:12.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:44:12.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:44:12.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:44:12.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:44:12.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:44:12.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:44:12.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:44:12.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:44:12.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e240824a-31a7-42ba-beab-44fc37418d76 2026-02-17T11:44:12.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 e240824a-31a7-42ba-beab-44fc37418d76' 2026-02-17T11:44:12.564 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 e240824a-31a7-42ba-beab-44fc37418d76 2026-02-17T11:44:12.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:44:12.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCMVJRpn5+cIRAAsJVAOJ7xF0Hlo9uBB7g91Q== 2026-02-17T11:44:12.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCMVJRpn5+cIRAAsJVAOJ7xF0Hlo9uBB7g91Q=="}' 2026-02-17T11:44:12.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e240824a-31a7-42ba-beab-44fc37418d76 -i td/osd-scrub-repair/2/new.json 2026-02-17T11:44:12.730 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:44:12.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T11:44:12.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCMVJRpn5+cIRAAsJVAOJ7xF0Hlo9uBB7g91Q== --osd-uuid e240824a-31a7-42ba-beab-44fc37418d76 2026-02-17T11:44:12.756 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:12.748+0000 7fc0386e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:12.758 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:12.748+0000 7fc0386e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:12.759 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:12.748+0000 7fc0386e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:12.759 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:12.748+0000 7fc0386e08c0 -1 bdev(0x56372f46dc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T11:44:12.760 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:12.748+0000 7fc0386e08c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T11:44:15.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T11:44:15.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:44:15.019 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T11:44:15.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T11:44:15.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:44:15.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T11:44:15.215 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T11:44:15.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:44:15.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:44:15.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:44:15.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:44:15.238 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:15.228+0000 7fbd91b278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:15.240 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:15.228+0000 7fbd91b278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:15.241 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:15.232+0000 7fbd91b278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:15.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T11:44:15.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:44:15.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:44:15.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:44:15.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:44:15.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:15.366 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:44:15.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:44:15.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:15.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:44:15.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:16.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:16.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:16.523 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:44:16.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:44:16.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:16.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:44:16.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:16.742 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:16.732+0000 7fbd91b278c0 -1 Falling back to public interface 2026-02-17T11:44:17.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:17.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:17.678 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:44:17.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:44:17.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:17.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:44:17.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:18.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:18.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:18.833 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:44:18.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:44:18.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:18.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:44:18.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:19.813 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:19.803+0000 7fbd91b278c0 -1 osd.2 0 log_to_monitors true 2026-02-17T11:44:19.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:19.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:19.991 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:44:19.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:44:19.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:19.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:44:20.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:21.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:21.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:21.151 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:44:21.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:44:21.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:21.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:44:21.304 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3495402496,v1:127.0.0.1:6819/3495402496] [v2:127.0.0.1:6820/3495402496,v1:127.0.0.1:6821/3495402496] exists,up e240824a-31a7-42ba-beab-44fc37418d76 2026-02-17T11:44:21.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:44:21.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:44:21.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:44:21.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:44:21.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 3 2026-02-17T11:44:21.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:44:21.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:44:21.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T11:44:21.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:44:21.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:44:21.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:44:21.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:44:21.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:44:21.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:44:21.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:44:21.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:44:21.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:44:21.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:44:21.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:44:21.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:44:21.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:44:21.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:44:21.306 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:44:21.306 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:44:21.306 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:44:21.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:44:21.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:44:21.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:44:21.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:44:21.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:44:21.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:44:21.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:44:21.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:44:21.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:44:21.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:44:21.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:44:21.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:44:21.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:44:21.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:44:21.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:44:21.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3866ec67-d79b-45ae-83d0-612ac11a1b5b 2026-02-17T11:44:21.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 3866ec67-d79b-45ae-83d0-612ac11a1b5b' 2026-02-17T11:44:21.309 INFO:tasks.workunit.client.0.trial128.stdout:add osd3 3866ec67-d79b-45ae-83d0-612ac11a1b5b 2026-02-17T11:44:21.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:44:21.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCVVJRp+xS/EhAAE6yqGsHRgCtM3nRMbkBnzw== 2026-02-17T11:44:21.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCVVJRp+xS/EhAAE6yqGsHRgCtM3nRMbkBnzw=="}' 2026-02-17T11:44:21.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3866ec67-d79b-45ae-83d0-612ac11a1b5b -i td/osd-scrub-repair/3/new.json 2026-02-17T11:44:21.479 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:44:21.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-17T11:44:21.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCVVJRp+xS/EhAAE6yqGsHRgCtM3nRMbkBnzw== --osd-uuid 3866ec67-d79b-45ae-83d0-612ac11a1b5b 2026-02-17T11:44:21.505 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:21.495+0000 7f19133df8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:21.506 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:21.495+0000 7f19133df8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:21.507 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:21.495+0000 7f19133df8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:21.508 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:21.495+0000 7f19133df8c0 -1 bdev(0x556d0c481c00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-17T11:44:21.508 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:21.495+0000 7f19133df8c0 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-17T11:44:23.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-17T11:44:23.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:44:23.810 INFO:tasks.workunit.client.0.trial128.stdout:adding osd3 key to auth repository 2026-02-17T11:44:23.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T11:44:23.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:44:24.014 INFO:tasks.workunit.client.0.trial128.stdout:start osd.3 2026-02-17T11:44:24.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T11:44:24.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:44:24.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:44:24.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:44:24.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:44:24.037 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:24.027+0000 7f596a8358c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:24.039 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:24.027+0000 7f596a8358c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:24.039 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:24.027+0000 7f596a8358c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:24.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T11:44:24.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:44:24.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:44:24.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:44:24.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:44:24.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:24.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:44:24.173 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:44:24.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:24.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:44:24.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:25.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:25.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:25.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:44:25.328 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:44:25.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:25.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:44:25.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:25.562 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:25.551+0000 7f596a8358c0 -1 Falling back to public interface 2026-02-17T11:44:26.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:26.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:26.481 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:44:26.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:44:26.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:26.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:44:26.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:27.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:27.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:27.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:44:27.636 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:44:27.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:27.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:44:27.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:28.553 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:28.543+0000 7f596a8358c0 -1 osd.3 0 log_to_monitors true 2026-02-17T11:44:28.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:28.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:28.792 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:44:28.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:44:28.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:28.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:44:28.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:29.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:29.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:29.940 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:44:29.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:44:29.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:29.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:44:30.093 INFO:tasks.workunit.client.0.trial128.stdout:osd.3 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1594907598,v1:127.0.0.1:6827/1594907598] [v2:127.0.0.1:6828/1594907598,v1:127.0.0.1:6829/1594907598] exists,up 3866ec67-d79b-45ae-83d0-612ac11a1b5b 2026-02-17T11:44:30.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:44:30.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:44:30.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:44:30.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:44:30.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 4 2026-02-17T11:44:30.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=4 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/4 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/4' 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/4/journal' 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:44:30.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:44:30.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:44:30.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:44:30.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:44:30.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:44:30.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:44:30.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:44:30.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:44:30.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:44:30.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:44:30.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:44:30.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:44:30.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:44:30.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/4 2026-02-17T11:44:30.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:44:30.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=40ff1bc5-e6dd-4ad8-aeb1-ab63191ee678 2026-02-17T11:44:30.097 INFO:tasks.workunit.client.0.trial128.stdout:add osd4 40ff1bc5-e6dd-4ad8-aeb1-ab63191ee678 2026-02-17T11:44:30.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd4 40ff1bc5-e6dd-4ad8-aeb1-ab63191ee678' 2026-02-17T11:44:30.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:44:30.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCeVJRpmcMfBhAAAvMEdJUoGGc2a2/2zUWkiw== 2026-02-17T11:44:30.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCeVJRpmcMfBhAAAvMEdJUoGGc2a2/2zUWkiw=="}' 2026-02-17T11:44:30.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 40ff1bc5-e6dd-4ad8-aeb1-ab63191ee678 -i td/osd-scrub-repair/4/new.json 2026-02-17T11:44:30.266 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:44:30.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/4/new.json 2026-02-17T11:44:30.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 4 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/4 --osd-journal=td/osd-scrub-repair/4/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCeVJRpmcMfBhAAAvMEdJUoGGc2a2/2zUWkiw== --osd-uuid 40ff1bc5-e6dd-4ad8-aeb1-ab63191ee678 2026-02-17T11:44:30.300 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:30.287+0000 7faff38048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:30.302 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:30.291+0000 7faff38048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:30.303 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:30.291+0000 7faff38048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:30.303 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:30.291+0000 7faff38048c0 -1 bdev(0x55b4e4045c00 td/osd-scrub-repair/4/block) open stat got: (1) Operation not permitted 2026-02-17T11:44:30.303 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:30.291+0000 7faff38048c0 -1 bluestore(td/osd-scrub-repair/4) _read_fsid unparsable uuid 2026-02-17T11:44:32.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/4/keyring 2026-02-17T11:44:32.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:44:32.572 INFO:tasks.workunit.client.0.trial128.stdout:adding osd4 key to auth repository 2026-02-17T11:44:32.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd4 key to auth repository 2026-02-17T11:44:32.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:44:32.773 INFO:tasks.workunit.client.0.trial128.stdout:start osd.4 2026-02-17T11:44:32.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.4 2026-02-17T11:44:32.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 4 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/4 --osd-journal=td/osd-scrub-repair/4/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:44:32.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:44:32.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:44:32.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:44:32.795 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:32.783+0000 7fc17563b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:32.797 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:32.787+0000 7fc17563b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:32.798 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:32.787+0000 7fc17563b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:32.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 4 2026-02-17T11:44:32.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:44:32.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=4 2026-02-17T11:44:32.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:44:32.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:44:32.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:32.924 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:44:32.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:44:32.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:32.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T11:44:33.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:34.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:34.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:34.080 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:44:34.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:44:34.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:34.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T11:44:34.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:34.314 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:34.303+0000 7fc17563b8c0 -1 Falling back to public interface 2026-02-17T11:44:35.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:35.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:35.234 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:44:35.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:44:35.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:35.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T11:44:35.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:36.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:36.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:36.390 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:44:36.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:44:36.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:36.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T11:44:36.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:37.343 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:37.331+0000 7fc17563b8c0 -1 osd.4 0 log_to_monitors true 2026-02-17T11:44:37.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:37.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:37.543 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:44:37.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:44:37.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:37.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T11:44:37.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:38.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:38.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:38.699 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:44:38.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:44:38.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:38.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T11:44:38.852 INFO:tasks.workunit.client.0.trial128.stdout:osd.4 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/4061338497,v1:127.0.0.1:6835/4061338497] [v2:127.0.0.1:6836/4061338497,v1:127.0.0.1:6837/4061338497] exists,up 40ff1bc5-e6dd-4ad8-aeb1-ab63191ee678 2026-02-17T11:44:38.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:44:38.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:44:38.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:44:38.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:44:38.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 5 2026-02-17T11:44:38.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:44:38.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:44:38.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=5 2026-02-17T11:44:38.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:44:38.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:44:38.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:44:38.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:44:38.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:44:38.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:44:38.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T11:44:38.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T11:44:38.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:44:38.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:44:38.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:44:38.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:44:38.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:44:38.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:44:38.857 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:44:38.857 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:44:38.857 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:44:38.858 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:44:38.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:44:38.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:44:38.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:44:38.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:44:38.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:44:38.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:44:38.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:44:38.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:44:38.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:44:38.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:44:38.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:44:38.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:44:38.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T11:44:38.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:44:38.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=64f61c82-c6d6-4640-bd33-5fd8a48c3bfa 2026-02-17T11:44:38.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd5 64f61c82-c6d6-4640-bd33-5fd8a48c3bfa' 2026-02-17T11:44:38.860 INFO:tasks.workunit.client.0.trial128.stdout:add osd5 64f61c82-c6d6-4640-bd33-5fd8a48c3bfa 2026-02-17T11:44:38.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:44:38.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCmVJRpH9RiMxAAmPRooSz50ZwQ0VtWlqEwFA== 2026-02-17T11:44:38.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCmVJRpH9RiMxAAmPRooSz50ZwQ0VtWlqEwFA=="}' 2026-02-17T11:44:38.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 64f61c82-c6d6-4640-bd33-5fd8a48c3bfa -i td/osd-scrub-repair/5/new.json 2026-02-17T11:44:39.019 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:44:39.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/5/new.json 2026-02-17T11:44:39.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 5 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCmVJRpH9RiMxAAmPRooSz50ZwQ0VtWlqEwFA== --osd-uuid 64f61c82-c6d6-4640-bd33-5fd8a48c3bfa 2026-02-17T11:44:39.046 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:39.035+0000 7f58889a98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:39.048 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:39.035+0000 7f58889a98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:39.049 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:39.039+0000 7f58889a98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:39.049 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:39.039+0000 7f58889a98c0 -1 bdev(0x55f5ba321c00 td/osd-scrub-repair/5/block) open stat got: (1) Operation not permitted 2026-02-17T11:44:39.049 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:39.039+0000 7f58889a98c0 -1 bluestore(td/osd-scrub-repair/5) _read_fsid unparsable uuid 2026-02-17T11:44:41.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/5/keyring 2026-02-17T11:44:41.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:44:41.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd5 key to auth repository 2026-02-17T11:44:41.323 INFO:tasks.workunit.client.0.trial128.stdout:adding osd5 key to auth repository 2026-02-17T11:44:41.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:44:41.523 INFO:tasks.workunit.client.0.trial128.stdout:start osd.5 2026-02-17T11:44:41.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.5 2026-02-17T11:44:41.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 5 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:44:41.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:44:41.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:44:41.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:44:41.547 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:41.535+0000 7f57762238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:41.548 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:41.535+0000 7f57762238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:41.549 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:41.539+0000 7f57762238c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:41.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 5 2026-02-17T11:44:41.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:44:41.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T11:44:41.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:44:41.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:44:41.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:41.673 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:44:41.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:44:41.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:41.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:44:41.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:42.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:42.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:42.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:44:42.829 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:44:42.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:42.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:44:42.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:43.062 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:43.051+0000 7f57762238c0 -1 Falling back to public interface 2026-02-17T11:44:43.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:43.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:43.982 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:44:43.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:44:43.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:43.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:44:44.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:45.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:45.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:45.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:44:45.134 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:44:45.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:45.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:44:45.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:46.117 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:46.107+0000 7f57762238c0 -1 osd.5 0 log_to_monitors true 2026-02-17T11:44:46.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:46.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:46.287 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:44:46.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:44:46.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:46.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:44:46.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:47.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:47.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:47.445 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:44:47.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:44:47.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:47.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:44:47.596 INFO:tasks.workunit.client.0.trial128.stdout:osd.5 up in weight 1 up_from 27 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/145452953,v1:127.0.0.1:6843/145452953] [v2:127.0.0.1:6844/145452953,v1:127.0.0.1:6845/145452953] exists,up 64f61c82-c6d6-4640-bd33-5fd8a48c3bfa 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 6 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=6 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/6 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/6' 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/6/journal' 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:44:47.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:44:47.598 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:44:47.598 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:44:47.598 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:44:47.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:44:47.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:44:47.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:44:47.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:44:47.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:44:47.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:44:47.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:44:47.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:44:47.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:44:47.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:44:47.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:44:47.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:44:47.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:44:47.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/6 2026-02-17T11:44:47.599 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:44:47.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=16d884d3-1a18-42fe-83ab-8719b63e02b5 2026-02-17T11:44:47.600 INFO:tasks.workunit.client.0.trial128.stdout:add osd6 16d884d3-1a18-42fe-83ab-8719b63e02b5 2026-02-17T11:44:47.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd6 16d884d3-1a18-42fe-83ab-8719b63e02b5' 2026-02-17T11:44:47.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:44:47.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCvVJRpXb8dJBAArTej4TRgQ2OIAJTRCCW1LA== 2026-02-17T11:44:47.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCvVJRpXb8dJBAArTej4TRgQ2OIAJTRCCW1LA=="}' 2026-02-17T11:44:47.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 16d884d3-1a18-42fe-83ab-8719b63e02b5 -i td/osd-scrub-repair/6/new.json 2026-02-17T11:44:47.769 INFO:tasks.workunit.client.0.trial128.stdout:6 2026-02-17T11:44:47.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/6/new.json 2026-02-17T11:44:47.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 6 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/6 --osd-journal=td/osd-scrub-repair/6/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCvVJRpXb8dJBAArTej4TRgQ2OIAJTRCCW1LA== --osd-uuid 16d884d3-1a18-42fe-83ab-8719b63e02b5 2026-02-17T11:44:47.795 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:47.783+0000 7f84d78718c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:47.796 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:47.783+0000 7f84d78718c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:47.797 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:47.787+0000 7f84d78718c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:47.798 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:47.787+0000 7f84d78718c0 -1 bdev(0x55a1e341bc00 td/osd-scrub-repair/6/block) open stat got: (1) Operation not permitted 2026-02-17T11:44:47.799 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:47.787+0000 7f84d78718c0 -1 bluestore(td/osd-scrub-repair/6) _read_fsid unparsable uuid 2026-02-17T11:44:50.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/6/keyring 2026-02-17T11:44:50.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:44:50.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd6 key to auth repository 2026-02-17T11:44:50.096 INFO:tasks.workunit.client.0.trial128.stdout:adding osd6 key to auth repository 2026-02-17T11:44:50.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:44:50.275 INFO:tasks.workunit.client.0.trial128.stdout:start osd.6 2026-02-17T11:44:50.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.6 2026-02-17T11:44:50.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 6 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/6 --osd-journal=td/osd-scrub-repair/6/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:44:50.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:44:50.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:44:50.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:44:50.288 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:50.275+0000 7f2a67f248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:50.290 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:50.279+0000 7f2a67f248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:50.290 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:50.279+0000 7f2a67f248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:50.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 6 2026-02-17T11:44:50.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:44:50.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=6 2026-02-17T11:44:50.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:44:50.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:44:50.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:50.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:44:50.430 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:44:50.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:50.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T11:44:50.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:51.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:51.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:51.577 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:44:51.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:44:51.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:51.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T11:44:51.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:51.990 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:51.979+0000 7f2a67f248c0 -1 Falling back to public interface 2026-02-17T11:44:52.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:52.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:52.737 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:44:52.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:44:52.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:52.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T11:44:52.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:53.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:53.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:53.890 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:44:53.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:44:53.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:53.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T11:44:54.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:54.964 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:54.951+0000 7f2a67f248c0 -1 osd.6 0 log_to_monitors true 2026-02-17T11:44:55.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:55.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:55.046 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:44:55.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:44:55.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:55.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T11:44:55.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:44:56.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:44:56.201 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:44:56.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:56.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:44:56.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:56.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T11:44:56.352 INFO:tasks.workunit.client.0.trial128.stdout:osd.6 up in weight 1 up_from 31 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3712583307,v1:127.0.0.1:6851/3712583307] [v2:127.0.0.1:6852/3712583307,v1:127.0.0.1:6853/3712583307] exists,up 16d884d3-1a18-42fe-83ab-8719b63e02b5 2026-02-17T11:44:56.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:44:56.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:44:56.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:44:56.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:44:56.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 7 2026-02-17T11:44:56.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:44:56.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:44:56.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=7 2026-02-17T11:44:56.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:44:56.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/7 2026-02-17T11:44:56.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:44:56.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:44:56.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:44:56.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:44:56.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/7' 2026-02-17T11:44:56.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/7/journal' 2026-02-17T11:44:56.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:44:56.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:44:56.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:44:56.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:44:56.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:44:56.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:44:56.358 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:44:56.358 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:44:56.358 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:44:56.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:44:56.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:44:56.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:44:56.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:44:56.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:44:56.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:44:56.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:44:56.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:44:56.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:44:56.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:44:56.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:44:56.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:44:56.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:44:56.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/7 2026-02-17T11:44:56.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:44:56.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=02d0a302-dcf6-4f63-b5e6-cdbaf74ab29b 2026-02-17T11:44:56.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd7 02d0a302-dcf6-4f63-b5e6-cdbaf74ab29b' 2026-02-17T11:44:56.360 INFO:tasks.workunit.client.0.trial128.stdout:add osd7 02d0a302-dcf6-4f63-b5e6-cdbaf74ab29b 2026-02-17T11:44:56.361 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:44:56.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQC4VJRpLcySFRAA8zKn1K6JIIE3NAqowIFpOA== 2026-02-17T11:44:56.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQC4VJRpLcySFRAA8zKn1K6JIIE3NAqowIFpOA=="}' 2026-02-17T11:44:56.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 02d0a302-dcf6-4f63-b5e6-cdbaf74ab29b -i td/osd-scrub-repair/7/new.json 2026-02-17T11:44:56.525 INFO:tasks.workunit.client.0.trial128.stdout:7 2026-02-17T11:44:56.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/7/new.json 2026-02-17T11:44:56.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 7 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/7 --osd-journal=td/osd-scrub-repair/7/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC4VJRpLcySFRAA8zKn1K6JIIE3NAqowIFpOA== --osd-uuid 02d0a302-dcf6-4f63-b5e6-cdbaf74ab29b 2026-02-17T11:44:56.550 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:56.538+0000 7f764428e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:56.552 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:56.538+0000 7f764428e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:56.553 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:56.542+0000 7f764428e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:56.553 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:56.542+0000 7f764428e8c0 -1 bdev(0x561b3d479c00 td/osd-scrub-repair/7/block) open stat got: (1) Operation not permitted 2026-02-17T11:44:56.554 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:56.542+0000 7f764428e8c0 -1 bluestore(td/osd-scrub-repair/7) _read_fsid unparsable uuid 2026-02-17T11:44:58.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/7/keyring 2026-02-17T11:44:58.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:44:58.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd7 key to auth repository 2026-02-17T11:44:58.848 INFO:tasks.workunit.client.0.trial128.stdout:adding osd7 key to auth repository 2026-02-17T11:44:58.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:44:59.049 INFO:tasks.workunit.client.0.trial128.stdout:start osd.7 2026-02-17T11:44:59.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.7 2026-02-17T11:44:59.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 7 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/7 --osd-journal=td/osd-scrub-repair/7/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:44:59.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:44:59.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:44:59.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:44:59.071 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:59.058+0000 7f66adbdb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:59.073 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:59.062+0000 7f66adbdb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:59.074 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:44:59.062+0000 7f66adbdb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:44:59.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 7 2026-02-17T11:44:59.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:44:59.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=7 2026-02-17T11:44:59.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:44:59.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:44:59.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:44:59.199 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:44:59.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:44:59.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:44:59.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T11:44:59.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:00.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:00.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:00.356 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:45:00.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:45:00.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:00.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T11:45:00.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:00.586 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:00.574+0000 7f66adbdb8c0 -1 Falling back to public interface 2026-02-17T11:45:01.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:01.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:01.510 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:45:01.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:45:01.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:01.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T11:45:01.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:02.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:02.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:02.662 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:45:02.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:45:02.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:02.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T11:45:02.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:03.578 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:03.566+0000 7f66adbdb8c0 -1 osd.7 0 log_to_monitors true 2026-02-17T11:45:03.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:03.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:03.816 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:45:03.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:45:03.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:03.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T11:45:03.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:04.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:04.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:04.973 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:45:04.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:45:04.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:04.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T11:45:05.126 INFO:tasks.workunit.client.0.trial128.stdout:osd.7 up in weight 1 up_from 35 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/2859770457,v1:127.0.0.1:6859/2859770457] [v2:127.0.0.1:6860/2859770457,v1:127.0.0.1:6861/2859770457] exists,up 02d0a302-dcf6-4f63-b5e6-cdbaf74ab29b 2026-02-17T11:45:05.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:45:05.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:45:05.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:45:05.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:45:05.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 8 2026-02-17T11:45:05.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:45:05.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:45:05.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=8 2026-02-17T11:45:05.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:45:05.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/8 2026-02-17T11:45:05.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:45:05.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:45:05.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:45:05.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:45:05.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/8' 2026-02-17T11:45:05.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/8/journal' 2026-02-17T11:45:05.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:45:05.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:45:05.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:45:05.132 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:45:05.132 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:45:05.132 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:45:05.132 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:45:05.132 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:45:05.132 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:45:05.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:45:05.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:45:05.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:45:05.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:45:05.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:45:05.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:45:05.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:45:05.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:45:05.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:45:05.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:45:05.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:45:05.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:45:05.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:45:05.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/8 2026-02-17T11:45:05.134 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:45:05.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=aaa0550c-7cff-4992-8069-db2fe1dfca5d 2026-02-17T11:45:05.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd8 aaa0550c-7cff-4992-8069-db2fe1dfca5d' 2026-02-17T11:45:05.135 INFO:tasks.workunit.client.0.trial128.stdout:add osd8 aaa0550c-7cff-4992-8069-db2fe1dfca5d 2026-02-17T11:45:05.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:45:05.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDBVJRp04USCBAApDTVLOEPqoiwH41GyB5R9g== 2026-02-17T11:45:05.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDBVJRp04USCBAApDTVLOEPqoiwH41GyB5R9g=="}' 2026-02-17T11:45:05.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new aaa0550c-7cff-4992-8069-db2fe1dfca5d -i td/osd-scrub-repair/8/new.json 2026-02-17T11:45:05.302 INFO:tasks.workunit.client.0.trial128.stdout:8 2026-02-17T11:45:05.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/8/new.json 2026-02-17T11:45:05.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 8 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/8 --osd-journal=td/osd-scrub-repair/8/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDBVJRp04USCBAApDTVLOEPqoiwH41GyB5R9g== --osd-uuid aaa0550c-7cff-4992-8069-db2fe1dfca5d 2026-02-17T11:45:05.327 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:05.314+0000 7ffac24ff8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:05.329 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:05.314+0000 7ffac24ff8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:05.329 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:05.318+0000 7ffac24ff8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:05.330 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:05.318+0000 7ffac24ff8c0 -1 bdev(0x55b7684c9c00 td/osd-scrub-repair/8/block) open stat got: (1) Operation not permitted 2026-02-17T11:45:05.330 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:05.318+0000 7ffac24ff8c0 -1 bluestore(td/osd-scrub-repair/8) _read_fsid unparsable uuid 2026-02-17T11:45:07.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/8/keyring 2026-02-17T11:45:07.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:45:07.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd8 key to auth repository 2026-02-17T11:45:07.858 INFO:tasks.workunit.client.0.trial128.stdout:adding osd8 key to auth repository 2026-02-17T11:45:07.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:45:08.056 INFO:tasks.workunit.client.0.trial128.stdout:start osd.8 2026-02-17T11:45:08.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.8 2026-02-17T11:45:08.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 8 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/8 --osd-journal=td/osd-scrub-repair/8/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:45:08.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:45:08.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:45:08.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:45:08.071 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:08.058+0000 7ff06f5d28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:08.072 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:08.058+0000 7ff06f5d28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:08.073 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:08.062+0000 7ff06f5d28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:08.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 8 2026-02-17T11:45:08.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:45:08.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=8 2026-02-17T11:45:08.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:45:08.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:45:08.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:08.207 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:45:08.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:45:08.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:08.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T11:45:08.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:09.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:09.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:09.362 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:45:09.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:45:09.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:09.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T11:45:09.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:09.594 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:09.582+0000 7ff06f5d28c0 -1 Falling back to public interface 2026-02-17T11:45:10.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:10.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:10.515 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:45:10.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:45:10.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:10.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T11:45:10.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:11.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:11.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:11.670 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:45:11.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:45:11.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:11.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T11:45:11.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:12.819 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:12.806+0000 7ff06f5d28c0 -1 osd.8 0 log_to_monitors true 2026-02-17T11:45:12.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:12.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:12.823 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:45:12.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:45:12.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:12.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T11:45:12.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:13.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:13.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:13.978 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:45:13.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:45:13.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:13.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T11:45:14.136 INFO:tasks.workunit.client.0.trial128.stdout:osd.8 up in weight 1 up_from 39 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/1279797037,v1:127.0.0.1:6867/1279797037] [v2:127.0.0.1:6868/1279797037,v1:127.0.0.1:6869/1279797037] exists,up aaa0550c-7cff-4992-8069-db2fe1dfca5d 2026-02-17T11:45:14.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:45:14.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:45:14.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:45:14.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:45:14.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 9 2026-02-17T11:45:14.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:45:14.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:45:14.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=9 2026-02-17T11:45:14.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:45:14.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:45:14.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:45:14.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:45:14.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:45:14.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:45:14.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T11:45:14.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T11:45:14.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:45:14.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:45:14.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:45:14.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:45:14.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:45:14.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:45:14.142 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:45:14.142 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:45:14.142 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:45:14.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:45:14.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:45:14.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:45:14.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:45:14.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:45:14.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:45:14.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:45:14.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:45:14.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:45:14.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:45:14.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:45:14.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:45:14.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:45:14.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T11:45:14.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:45:14.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e8ade1d3-5ef4-430b-bf60-7a9cf8ad1fa4 2026-02-17T11:45:14.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd9 e8ade1d3-5ef4-430b-bf60-7a9cf8ad1fa4' 2026-02-17T11:45:14.144 INFO:tasks.workunit.client.0.trial128.stdout:add osd9 e8ade1d3-5ef4-430b-bf60-7a9cf8ad1fa4 2026-02-17T11:45:14.145 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:45:14.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDKVJRpz++WCBAA2bky8R+XU/X+wiVtfgvi8g== 2026-02-17T11:45:14.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDKVJRpz++WCBAA2bky8R+XU/X+wiVtfgvi8g=="}' 2026-02-17T11:45:14.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e8ade1d3-5ef4-430b-bf60-7a9cf8ad1fa4 -i td/osd-scrub-repair/9/new.json 2026-02-17T11:45:14.311 INFO:tasks.workunit.client.0.trial128.stdout:9 2026-02-17T11:45:14.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/9/new.json 2026-02-17T11:45:14.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 9 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDKVJRpz++WCBAA2bky8R+XU/X+wiVtfgvi8g== --osd-uuid e8ade1d3-5ef4-430b-bf60-7a9cf8ad1fa4 2026-02-17T11:45:14.337 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:14.326+0000 7fdc259dc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:14.338 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:14.326+0000 7fdc259dc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:14.340 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:14.326+0000 7fdc259dc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:14.340 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:14.326+0000 7fdc259dc8c0 -1 bdev(0x55b41b8c5c00 td/osd-scrub-repair/9/block) open stat got: (1) Operation not permitted 2026-02-17T11:45:14.340 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:14.326+0000 7fdc259dc8c0 -1 bluestore(td/osd-scrub-repair/9) _read_fsid unparsable uuid 2026-02-17T11:45:16.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/9/keyring 2026-02-17T11:45:16.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:45:16.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd9 key to auth repository 2026-02-17T11:45:16.643 INFO:tasks.workunit.client.0.trial128.stdout:adding osd9 key to auth repository 2026-02-17T11:45:16.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/9/keyring auth add osd.9 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:45:16.840 INFO:tasks.workunit.client.0.trial128.stdout:start osd.9 2026-02-17T11:45:16.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.9 2026-02-17T11:45:16.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 9 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:45:16.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:45:16.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:45:16.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:45:16.864 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:16.850+0000 7fc0994958c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:16.865 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:16.854+0000 7fc0994958c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:16.866 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:16.854+0000 7fc0994958c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:16.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 9 2026-02-17T11:45:16.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:45:16.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T11:45:16.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:45:16.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:45:16.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:16.991 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:45:16.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:45:16.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:16.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:45:17.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:18.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:18.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:18.146 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:45:18.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:45:18.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:18.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:45:18.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:18.342 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:18.330+0000 7fc0994958c0 -1 Falling back to public interface 2026-02-17T11:45:19.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:19.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:19.301 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:45:19.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:45:19.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:19.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:45:19.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:20.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:20.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:20.454 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:45:20.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:45:20.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:20.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:45:20.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:21.398 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:21.386+0000 7fc0994958c0 -1 osd.9 0 log_to_monitors true 2026-02-17T11:45:21.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:21.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:21.607 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:45:21.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:45:21.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:21.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:45:21.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:22.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:22.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:22.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:45:22.762 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:45:22.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:22.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:45:22.911 INFO:tasks.workunit.client.0.trial128.stdout:osd.9 up in weight 1 up_from 44 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6874/3260503144,v1:127.0.0.1:6875/3260503144] [v2:127.0.0.1:6876/3260503144,v1:127.0.0.1:6877/3260503144] exists,up e8ade1d3-5ef4-430b-bf60-7a9cf8ad1fa4 2026-02-17T11:45:22.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:45:22.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:45:22.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:45:22.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:834: corrupt_and_repair_lrc: create_rbd_pool 2026-02-17T11:45:22.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T11:45:23.059 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' does not exist 2026-02-17T11:45:23.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T11:45:23.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T11:45:23.286 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T11:45:23.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:45:24.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T11:45:24.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:835: corrupt_and_repair_lrc: wait_for_clean 2026-02-17T11:45:24.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:45:24.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:45:24.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:45:24.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:45:24.576 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:45:24.576 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:45:24.576 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:45:24.576 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:45:24.576 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:45:24.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:45:24.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:45:24.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:45:24.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:45:24.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:45:24.689 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:45:24.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:45:24.843 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:45:24.843 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:45:24.843 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:45:24.843 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:45:24.843 INFO:tasks.workunit.client.0.trial128.stderr:5 2026-02-17T11:45:24.843 INFO:tasks.workunit.client.0.trial128.stderr:6 2026-02-17T11:45:24.843 INFO:tasks.workunit.client.0.trial128.stderr:7 2026-02-17T11:45:24.843 INFO:tasks.workunit.client.0.trial128.stderr:8 2026-02-17T11:45:24.843 INFO:tasks.workunit.client.0.trial128.stderr:9' 2026-02-17T11:45:24.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:45:24.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:24.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:45:24.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836498 2026-02-17T11:45:24.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836498 2026-02-17T11:45:24.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498' 2026-02-17T11:45:24.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:24.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:45:24.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705680 2026-02-17T11:45:24.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705680 2026-02-17T11:45:24.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680' 2026-02-17T11:45:24.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:24.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:45:25.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542158 2026-02-17T11:45:25.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542158 2026-02-17T11:45:25.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-60129542158' 2026-02-17T11:45:25.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:25.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:45:25.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=81604378637 2026-02-17T11:45:25.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 81604378637 2026-02-17T11:45:25.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-60129542158 3-81604378637' 2026-02-17T11:45:25.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:25.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:45:25.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247819 2026-02-17T11:45:25.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247819 2026-02-17T11:45:25.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-60129542158 3-81604378637 4-98784247819' 2026-02-17T11:45:25.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:25.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:45:25.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964117001 2026-02-17T11:45:25.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964117001 2026-02-17T11:45:25.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-60129542158 3-81604378637 4-98784247819 5-115964117001' 2026-02-17T11:45:25.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:25.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:45:25.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986183 2026-02-17T11:45:25.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986183 2026-02-17T11:45:25.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-60129542158 3-81604378637 4-98784247819 5-115964117001 6-133143986183' 2026-02-17T11:45:25.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:25.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:45:25.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855366 2026-02-17T11:45:25.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855366 2026-02-17T11:45:25.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-60129542158 3-81604378637 4-98784247819 5-115964117001 6-133143986183 7-150323855366' 2026-02-17T11:45:25.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:25.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:45:25.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724549 2026-02-17T11:45:25.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724549 2026-02-17T11:45:25.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-60129542158 3-81604378637 4-98784247819 5-115964117001 6-133143986183 7-150323855366 8-167503724549' 2026-02-17T11:45:25.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:25.488 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:45:25.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561026 2026-02-17T11:45:25.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561026 2026-02-17T11:45:25.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-60129542158 3-81604378637 4-98784247819 5-115964117001 6-133143986183 7-150323855366 8-167503724549 9-188978561026' 2026-02-17T11:45:25.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:25.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836498 2026-02-17T11:45:25.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:25.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:45:25.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836498 2026-02-17T11:45:25.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:25.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836498 2026-02-17T11:45:25.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836498' 2026-02-17T11:45:25.563 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836498 2026-02-17T11:45:25.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:45:25.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836497 -lt 21474836498 2026-02-17T11:45:25.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:45:26.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:45:26.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:45:26.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836497 -lt 21474836498 2026-02-17T11:45:26.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:45:27.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:45:27.877 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:45:28.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836498 -lt 21474836498 2026-02-17T11:45:28.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:28.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705680 2026-02-17T11:45:28.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:28.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:45:28.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705680 2026-02-17T11:45:28.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:28.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705680 2026-02-17T11:45:28.037 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705680 2026-02-17T11:45:28.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705680' 2026-02-17T11:45:28.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:45:28.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705680 -lt 38654705680 2026-02-17T11:45:28.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:28.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542158 2026-02-17T11:45:28.198 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:28.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:45:28.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542158 2026-02-17T11:45:28.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:28.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542158 2026-02-17T11:45:28.201 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542158 2026-02-17T11:45:28.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542158' 2026-02-17T11:45:28.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:45:28.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542159 -lt 60129542158 2026-02-17T11:45:28.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:28.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-81604378637 2026-02-17T11:45:28.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:28.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:45:28.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-81604378637 2026-02-17T11:45:28.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:28.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=81604378637 2026-02-17T11:45:28.358 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.3 seq 81604378637 2026-02-17T11:45:28.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 81604378637' 2026-02-17T11:45:28.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:45:28.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 81604378637 -lt 81604378637 2026-02-17T11:45:28.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:28.511 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247819 2026-02-17T11:45:28.511 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:28.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:45:28.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247819 2026-02-17T11:45:28.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:28.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247819 2026-02-17T11:45:28.515 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.4 seq 98784247819 2026-02-17T11:45:28.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247819' 2026-02-17T11:45:28.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:45:28.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247819 -lt 98784247819 2026-02-17T11:45:28.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:28.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-115964117001 2026-02-17T11:45:28.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:28.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:45:28.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-115964117001 2026-02-17T11:45:28.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:28.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964117001 2026-02-17T11:45:28.682 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.5 seq 115964117001 2026-02-17T11:45:28.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 115964117001' 2026-02-17T11:45:28.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:45:28.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964117001 -lt 115964117001 2026-02-17T11:45:28.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:28.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986183 2026-02-17T11:45:28.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:28.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:45:28.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986183 2026-02-17T11:45:28.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:28.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986183 2026-02-17T11:45:28.835 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.6 seq 133143986183 2026-02-17T11:45:28.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986183' 2026-02-17T11:45:28.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:45:28.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986184 -lt 133143986183 2026-02-17T11:45:28.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:28.992 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-150323855366 2026-02-17T11:45:28.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:28.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:45:28.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-150323855366 2026-02-17T11:45:28.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:28.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855366 2026-02-17T11:45:28.996 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.7 seq 150323855366 2026-02-17T11:45:28.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 150323855366' 2026-02-17T11:45:28.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:45:29.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855366 -lt 150323855366 2026-02-17T11:45:29.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:29.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724549 2026-02-17T11:45:29.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:29.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:45:29.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724549 2026-02-17T11:45:29.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:29.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724549 2026-02-17T11:45:29.160 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.8 seq 167503724549 2026-02-17T11:45:29.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724549' 2026-02-17T11:45:29.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:45:29.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724549 -lt 167503724549 2026-02-17T11:45:29.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:29.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-188978561026 2026-02-17T11:45:29.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:29.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:45:29.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-188978561026 2026-02-17T11:45:29.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:29.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561026 2026-02-17T11:45:29.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 188978561026' 2026-02-17T11:45:29.327 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.9 seq 188978561026 2026-02-17T11:45:29.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:45:29.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561027 -lt 188978561026 2026-02-17T11:45:29.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:45:29.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:45:29.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:45:29.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T11:45:29.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:45:29.664 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:45:29.664 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:45:29.664 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:45:29.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:45:29.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:45:29.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:45:29.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T11:45:29.813 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:45:29.813 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:45:29.813 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:45:29.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T11:45:29.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:45:29.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:45:29.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:837: corrupt_and_repair_lrc: create_ec_pool ecpool false k=4 m=2 l=3 plugin=lrc 2026-02-17T11:45:29.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T11:45:29.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T11:45:29.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=false 2026-02-17T11:45:29.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T11:45:29.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=4 m=2 l=3 plugin=lrc 2026-02-17T11:45:30.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T11:45:30.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T11:45:30.500 INFO:tasks.workunit.client.0.trial128.stderr:pool 'ecpool' created 2026-02-17T11:45:30.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:45:31.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' false = true ']' 2026-02-17T11:45:31.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T11:45:31.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:45:31.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:45:31.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:45:31.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:45:31.514 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:45:31.514 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:45:31.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:45:31.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:45:31.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:45:31.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:45:31.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:45:31.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:45:31.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:45:31.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:45:31.617 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:45:31.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:45:31.763 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:45:31.763 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:45:31.763 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:45:31.763 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:45:31.764 INFO:tasks.workunit.client.0.trial128.stderr:5 2026-02-17T11:45:31.764 INFO:tasks.workunit.client.0.trial128.stderr:6 2026-02-17T11:45:31.764 INFO:tasks.workunit.client.0.trial128.stderr:7 2026-02-17T11:45:31.764 INFO:tasks.workunit.client.0.trial128.stderr:8 2026-02-17T11:45:31.764 INFO:tasks.workunit.client.0.trial128.stderr:9' 2026-02-17T11:45:31.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:45:31.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:31.764 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:45:31.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836500 2026-02-17T11:45:31.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836500 2026-02-17T11:45:31.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500' 2026-02-17T11:45:31.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:31.837 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:45:31.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705682 2026-02-17T11:45:31.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705682 2026-02-17T11:45:31.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682' 2026-02-17T11:45:31.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:31.907 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:45:31.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542161 2026-02-17T11:45:31.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542161 2026-02-17T11:45:31.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-60129542161' 2026-02-17T11:45:31.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:31.980 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:45:32.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=81604378639 2026-02-17T11:45:32.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 81604378639 2026-02-17T11:45:32.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-60129542161 3-81604378639' 2026-02-17T11:45:32.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:32.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:45:32.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247821 2026-02-17T11:45:32.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247821 2026-02-17T11:45:32.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-60129542161 3-81604378639 4-98784247821' 2026-02-17T11:45:32.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:32.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:45:32.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964117003 2026-02-17T11:45:32.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964117003 2026-02-17T11:45:32.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-60129542161 3-81604378639 4-98784247821 5-115964117003' 2026-02-17T11:45:32.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:32.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:45:32.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986186 2026-02-17T11:45:32.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986186 2026-02-17T11:45:32.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-60129542161 3-81604378639 4-98784247821 5-115964117003 6-133143986186' 2026-02-17T11:45:32.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:32.267 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:45:32.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855368 2026-02-17T11:45:32.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855368 2026-02-17T11:45:32.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-60129542161 3-81604378639 4-98784247821 5-115964117003 6-133143986186 7-150323855368' 2026-02-17T11:45:32.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:32.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:45:32.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724551 2026-02-17T11:45:32.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724551 2026-02-17T11:45:32.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-60129542161 3-81604378639 4-98784247821 5-115964117003 6-133143986186 7-150323855368 8-167503724551' 2026-02-17T11:45:32.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:32.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:45:32.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561028 2026-02-17T11:45:32.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561028 2026-02-17T11:45:32.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-60129542161 3-81604378639 4-98784247821 5-115964117003 6-133143986186 7-150323855368 8-167503724551 9-188978561028' 2026-02-17T11:45:32.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:32.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836500 2026-02-17T11:45:32.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:32.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:45:32.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836500 2026-02-17T11:45:32.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:32.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836500 2026-02-17T11:45:32.483 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836500 2026-02-17T11:45:32.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836500' 2026-02-17T11:45:32.483 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:45:32.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836500 2026-02-17T11:45:32.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:45:33.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:45:33.632 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:45:33.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836500 -lt 21474836500 2026-02-17T11:45:33.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:33.786 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705682 2026-02-17T11:45:33.786 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:33.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:45:33.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705682 2026-02-17T11:45:33.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:33.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705682 2026-02-17T11:45:33.791 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705682 2026-02-17T11:45:33.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705682' 2026-02-17T11:45:33.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:45:33.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705683 -lt 38654705682 2026-02-17T11:45:33.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:33.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542161 2026-02-17T11:45:33.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:33.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:45:33.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542161 2026-02-17T11:45:33.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:33.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542161 2026-02-17T11:45:33.945 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542161 2026-02-17T11:45:33.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542161' 2026-02-17T11:45:33.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:45:34.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542161 -lt 60129542161 2026-02-17T11:45:34.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:34.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-81604378639 2026-02-17T11:45:34.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:34.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:45:34.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-81604378639 2026-02-17T11:45:34.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:34.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=81604378639 2026-02-17T11:45:34.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 81604378639' 2026-02-17T11:45:34.108 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.3 seq 81604378639 2026-02-17T11:45:34.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:45:34.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 81604378639 -lt 81604378639 2026-02-17T11:45:34.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:34.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247821 2026-02-17T11:45:34.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:34.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:45:34.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247821 2026-02-17T11:45:34.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:34.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247821 2026-02-17T11:45:34.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247821' 2026-02-17T11:45:34.267 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.4 seq 98784247821 2026-02-17T11:45:34.267 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:45:34.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247821 -lt 98784247821 2026-02-17T11:45:34.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:34.429 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-115964117003 2026-02-17T11:45:34.429 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:34.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:45:34.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-115964117003 2026-02-17T11:45:34.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:34.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964117003 2026-02-17T11:45:34.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 115964117003' 2026-02-17T11:45:34.433 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.5 seq 115964117003 2026-02-17T11:45:34.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:45:34.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964117004 -lt 115964117003 2026-02-17T11:45:34.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:34.599 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986186 2026-02-17T11:45:34.599 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:34.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:45:34.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986186 2026-02-17T11:45:34.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:34.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986186 2026-02-17T11:45:34.602 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.6 seq 133143986186 2026-02-17T11:45:34.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986186' 2026-02-17T11:45:34.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:45:34.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986186 -lt 133143986186 2026-02-17T11:45:34.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:34.754 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-150323855368 2026-02-17T11:45:34.754 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:34.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:45:34.755 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-150323855368 2026-02-17T11:45:34.756 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:34.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855368 2026-02-17T11:45:34.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 150323855368' 2026-02-17T11:45:34.757 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.7 seq 150323855368 2026-02-17T11:45:34.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:45:34.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855369 -lt 150323855368 2026-02-17T11:45:34.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:34.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724551 2026-02-17T11:45:34.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:34.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:45:34.924 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724551 2026-02-17T11:45:34.924 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:34.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724551 2026-02-17T11:45:34.926 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.8 seq 167503724551 2026-02-17T11:45:34.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724551' 2026-02-17T11:45:34.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:45:35.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724552 -lt 167503724551 2026-02-17T11:45:35.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:35.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-188978561028 2026-02-17T11:45:35.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:35.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:45:35.085 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-188978561028 2026-02-17T11:45:35.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:35.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561028 2026-02-17T11:45:35.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 188978561028' 2026-02-17T11:45:35.088 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.9 seq 188978561028 2026-02-17T11:45:35.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:45:35.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561029 -lt 188978561028 2026-02-17T11:45:35.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:45:35.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:45:35.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:45:35.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:45:35.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:45:35.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:45:35.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:45:35.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:45:35.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:45:35.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:45:35.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:45:35.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:45:35.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:45:35.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:45:35.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:45:35.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:45:35.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:45:35.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:45:35.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T11:45:35.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:838: corrupt_and_repair_lrc: corrupt_and_repair_erasure_coded td/osd-scrub-repair ecpool 2026-02-17T11:45:35.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:248: corrupt_and_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T11:45:35.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:249: corrupt_and_repair_erasure_coded: local poolname=ecpool 2026-02-17T11:45:35.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:251: corrupt_and_repair_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-17T11:45:35.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:45:35.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:45:35.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T11:45:35.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:45:35.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:45:35.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:45:35.962 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:45:35.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:45:36.167 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:45:36.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:45:36.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:45:36.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T11:45:36.198 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_erasure_coded: get_primary ecpool SOMETHING 2026-02-17T11:45:36.198 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T11:45:36.198 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:45:36.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:45:36.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:45:36.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_erasure_coded: local primary=3 2026-02-17T11:45:36.349 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: get_osds ecpool SOMETHING 2026-02-17T11:45:36.349 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: sed -e s/3// 2026-02-17T11:45:36.350 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T11:45:36.350 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T11:45:36.350 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:45:36.350 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T11:45:36.499 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T11:45:36.500 INFO:tasks.workunit.client.0.trial128.stderr:5 2026-02-17T11:45:36.500 INFO:tasks.workunit.client.0.trial128.stderr:9 2026-02-17T11:45:36.500 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:45:36.500 INFO:tasks.workunit.client.0.trial128.stderr:6 2026-02-17T11:45:36.500 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:45:36.500 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:45:36.500 INFO:tasks.workunit.client.0.trial128.stderr:7' 2026-02-17T11:45:36.500 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 5 9 0 6 2 1 7 2026-02-17T11:45:36.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: osds=('5' '9' '0' '6' '2' '1' '7') 2026-02-17T11:45:36.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: local -a osds 2026-02-17T11:45:36.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:255: corrupt_and_repair_erasure_coded: local not_primary_first=5 2026-02-17T11:45:36.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:256: corrupt_and_repair_erasure_coded: local not_primary_second=9 2026-02-17T11:45:36.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:259: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 3 2026-02-17T11:45:36.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T11:45:36.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=ecpool 2026-02-17T11:45:36.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=3 2026-02-17T11:45:36.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:45:36.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:45:36.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:45:36.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T11:45:36.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:45:36.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:45:36.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:45:36.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:45:36.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T11:45:36.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:45:36.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T11:45:36.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:45:36.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:45:36.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:45:36.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:45:36.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:45:37.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:45:37.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:45:37.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:45:37.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:45:37.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T11:45:37.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:45:37.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:45:37.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T11:45:38.447 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#2:eb822e21:::SOMETHING:head# 2026-02-17T11:45:38.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T11:45:38.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:45:38.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:45:38.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T11:45:38.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:45:38.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:45:38.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:45:38.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:45:38.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:45:38.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:45:38.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:45:38.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:45:38.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:45:38.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:45:38.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:45:38.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:45:38.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:45:38.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:45:38.985 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:45:38.985 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:45:38.986 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:45:38.986 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:45:38.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:45:38.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:45:38.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:45:38.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:45:38.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:45:38.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:45:38.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:45:38.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:45:38.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:45:38.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:45:38.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:45:38.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T11:45:38.986 INFO:tasks.workunit.client.0.trial128.stderr:start osd.3 2026-02-17T11:45:38.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:45:38.987 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T11:45:38.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T11:45:38.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:45:38.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:45:38.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:45:39.010 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:39.001+0000 7f13ec4698c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:39.011 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:39.001+0000 7f13ec4698c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:39.012 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:39.001+0000 7f13ec4698c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:39.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T11:45:39.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:45:39.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:45:39.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:45:39.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:45:39.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:39.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:45:39.143 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:45:39.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:39.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:45:39.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:40.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:40.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:40.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:45:40.295 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:45:40.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:40.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:45:40.311 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:40.301+0000 7f13ec4698c0 -1 Falling back to public interface 2026-02-17T11:45:40.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:41.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:41.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:41.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:45:41.445 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:45:41.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:41.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:45:41.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:42.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:42.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:42.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:45:42.593 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:45:42.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:42.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:45:42.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:43.322 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:43.313+0000 7f13ec4698c0 -1 osd.3 61 log_to_monitors true 2026-02-17T11:45:43.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:43.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:43.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:45:43.748 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:45:43.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:43.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:45:43.893 INFO:tasks.workunit.client.0.trial128.stderr:osd.3 up in weight 1 up_from 65 up_thru 65 down_at 62 last_clean_interval [19,61) [v2:127.0.0.1:6826/3980148979,v1:127.0.0.1:6827/3980148979] [v2:127.0.0.1:6828/3980148979,v1:127.0.0.1:6829/3980148979] exists,up 3866ec67-d79b-45ae-83d0-612ac11a1b5b 2026-02-17T11:45:43.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:45:43.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:45:43.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:45:43.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:45:43.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:45:43.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:45:43.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:45:43.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:45:43.894 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:45:43.894 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:45:43.895 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:45:43.895 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:45:43.895 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:45:44.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:45:44.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:45:44.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:45:44.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:45:44.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:45:44.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:45:44.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:45:44.166 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:45:44.166 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:45:44.166 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:45:44.166 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:45:44.166 INFO:tasks.workunit.client.0.trial128.stderr:5 2026-02-17T11:45:44.166 INFO:tasks.workunit.client.0.trial128.stderr:6 2026-02-17T11:45:44.167 INFO:tasks.workunit.client.0.trial128.stderr:7 2026-02-17T11:45:44.167 INFO:tasks.workunit.client.0.trial128.stderr:8 2026-02-17T11:45:44.167 INFO:tasks.workunit.client.0.trial128.stderr:9' 2026-02-17T11:45:44.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:45:44.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:44.167 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:45:44.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836504 2026-02-17T11:45:44.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836504 2026-02-17T11:45:44.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504' 2026-02-17T11:45:44.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:44.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:45:44.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705686 2026-02-17T11:45:44.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705686 2026-02-17T11:45:44.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-38654705686' 2026-02-17T11:45:44.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:44.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:45:44.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542164 2026-02-17T11:45:44.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542164 2026-02-17T11:45:44.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-38654705686 2-60129542164' 2026-02-17T11:45:44.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:44.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:45:44.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874242 2026-02-17T11:45:44.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874242 2026-02-17T11:45:44.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-38654705686 2-60129542164 3-279172874242' 2026-02-17T11:45:44.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:44.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:45:44.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247825 2026-02-17T11:45:44.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247825 2026-02-17T11:45:44.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-38654705686 2-60129542164 3-279172874242 4-98784247825' 2026-02-17T11:45:44.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:44.524 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:45:44.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964117007 2026-02-17T11:45:44.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964117007 2026-02-17T11:45:44.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-38654705686 2-60129542164 3-279172874242 4-98784247825 5-115964117007' 2026-02-17T11:45:44.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:44.596 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:45:44.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986189 2026-02-17T11:45:44.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986189 2026-02-17T11:45:44.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-38654705686 2-60129542164 3-279172874242 4-98784247825 5-115964117007 6-133143986189' 2026-02-17T11:45:44.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:44.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:45:44.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855371 2026-02-17T11:45:44.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855371 2026-02-17T11:45:44.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-38654705686 2-60129542164 3-279172874242 4-98784247825 5-115964117007 6-133143986189 7-150323855371' 2026-02-17T11:45:44.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:44.737 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:45:44.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724555 2026-02-17T11:45:44.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724555 2026-02-17T11:45:44.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-38654705686 2-60129542164 3-279172874242 4-98784247825 5-115964117007 6-133143986189 7-150323855371 8-167503724555' 2026-02-17T11:45:44.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:44.812 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:45:44.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561032 2026-02-17T11:45:44.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561032 2026-02-17T11:45:44.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836504 1-38654705686 2-60129542164 3-279172874242 4-98784247825 5-115964117007 6-133143986189 7-150323855371 8-167503724555 9-188978561032' 2026-02-17T11:45:44.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:44.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836504 2026-02-17T11:45:44.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:44.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:45:44.884 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836504 2026-02-17T11:45:44.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:44.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836504 2026-02-17T11:45:44.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836504' 2026-02-17T11:45:44.886 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836504 2026-02-17T11:45:44.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:45:45.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836504 -lt 21474836504 2026-02-17T11:45:45.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:45.043 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705686 2026-02-17T11:45:45.043 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:45.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:45:45.046 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705686 2026-02-17T11:45:45.046 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:45.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705686 2026-02-17T11:45:45.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705686' 2026-02-17T11:45:45.047 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705686 2026-02-17T11:45:45.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:45:45.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705686 -lt 38654705686 2026-02-17T11:45:45.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:45.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542164 2026-02-17T11:45:45.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:45.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:45:45.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542164 2026-02-17T11:45:45.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:45.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542164 2026-02-17T11:45:45.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542164' 2026-02-17T11:45:45.209 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542164 2026-02-17T11:45:45.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:45:45.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542164 -lt 60129542164 2026-02-17T11:45:45.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:45.374 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874242 2026-02-17T11:45:45.374 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:45.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:45:45.376 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874242 2026-02-17T11:45:45.376 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:45.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874242 2026-02-17T11:45:45.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874242' 2026-02-17T11:45:45.378 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.3 seq 279172874242 2026-02-17T11:45:45.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:45:45.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874242 -lt 279172874242 2026-02-17T11:45:45.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:45.531 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247825 2026-02-17T11:45:45.531 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:45.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:45:45.533 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247825 2026-02-17T11:45:45.533 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:45.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247825 2026-02-17T11:45:45.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247825' 2026-02-17T11:45:45.534 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.4 seq 98784247825 2026-02-17T11:45:45.535 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:45:45.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247825 -lt 98784247825 2026-02-17T11:45:45.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:45.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-115964117007 2026-02-17T11:45:45.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:45.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:45:45.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-115964117007 2026-02-17T11:45:45.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:45.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964117007 2026-02-17T11:45:45.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 115964117007' 2026-02-17T11:45:45.701 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.5 seq 115964117007 2026-02-17T11:45:45.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:45:45.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964117007 -lt 115964117007 2026-02-17T11:45:45.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:45.864 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986189 2026-02-17T11:45:45.864 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:45.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:45:45.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986189 2026-02-17T11:45:45.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:45.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986189 2026-02-17T11:45:45.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986189' 2026-02-17T11:45:45.868 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.6 seq 133143986189 2026-02-17T11:45:45.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:45:46.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986189 -lt 133143986189 2026-02-17T11:45:46.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:46.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-150323855371 2026-02-17T11:45:46.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:46.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:45:46.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-150323855371 2026-02-17T11:45:46.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:46.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855371 2026-02-17T11:45:46.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 150323855371' 2026-02-17T11:45:46.032 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.7 seq 150323855371 2026-02-17T11:45:46.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:45:46.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855372 -lt 150323855371 2026-02-17T11:45:46.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:46.198 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724555 2026-02-17T11:45:46.198 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:46.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:45:46.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724555 2026-02-17T11:45:46.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:46.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724555 2026-02-17T11:45:46.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724555' 2026-02-17T11:45:46.201 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.8 seq 167503724555 2026-02-17T11:45:46.201 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:45:46.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724555 -lt 167503724555 2026-02-17T11:45:46.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:46.350 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-188978561032 2026-02-17T11:45:46.350 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:46.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:45:46.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-188978561032 2026-02-17T11:45:46.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:46.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561032 2026-02-17T11:45:46.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 188978561032' 2026-02-17T11:45:46.354 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.9 seq 188978561032 2026-02-17T11:45:46.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:45:46.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561032 -lt 188978561032 2026-02-17T11:45:46.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:45:46.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:45:46.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:45:46.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:45:46.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:45:46.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:45:46.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:45:46.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:45:46.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:45:46.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:45:46.683 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:45:46.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:45:46.832 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:45:46.832 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:45:46.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:45:47.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:45:47.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:45:47.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:45:47.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-17T11:45:47.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:45:47.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:45:47.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:45:47.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:45:47.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=2.0 2026-02-17T11:45:47.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 2.0 2026-02-17T11:45:47.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T11:45:47.165 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T11:45:47.165 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:45:47.165 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:45:47.165 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:45:47.165 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:45:47.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:45:30.491408+0000 2026-02-17T11:45:47.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T11:45:47.457 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T11:45:47.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T11:45:30.491408+0000 2026-02-17T11:45:47.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:45:47.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:45:30.491408+0000 2026-02-17T11:45:47.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:45:47.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:45:47.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:45:47.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:45:47.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:45:47.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:45:47.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:45:47.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:45:47.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:45:30.491408+0000 '>' 2026-02-17T11:45:30.491408+0000 2026-02-17T11:45:47.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:45:48.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:45:48.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:45:48.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:45:48.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:45:48.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:45:48.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:45:48.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:45:48.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:45:30.491408+0000 '>' 2026-02-17T11:45:30.491408+0000 2026-02-17T11:45:48.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:45:49.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:45:49.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:45:49.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:45:49.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:45:49.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:45:49.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:45:49.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:45:49.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:45:48.306375+0000 '>' 2026-02-17T11:45:30.491408+0000 2026-02-17T11:45:49.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:45:49.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:45:49.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:45:49.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:45:49.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T11:45:49.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:45:49.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:45:49.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:45:49.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:45:49.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T11:45:49.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:45:49.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-17T11:45:49.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:45:49.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:45:49.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:45:49.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:45:49.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:45:51.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:45:51.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:45:51.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:45:51.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:45:51.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T11:45:51.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:45:51.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:45:51.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-17T11:45:51.552 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:45:51.552 INFO:tasks.workunit.client.0.trial128.stdout:hinfo_key 2026-02-17T11:45:51.552 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:45:52.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-17T11:45:52.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:45:52.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:45:52.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T11:45:52.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:45:52.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:45:52.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:45:52.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:45:52.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:45:52.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:45:52.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T11:45:52.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T11:45:52.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:45:52.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:45:52.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:45:52.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:45:52.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:45:52.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:45:52.094 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:45:52.094 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:45:52.094 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:45:52.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:45:52.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:45:52.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:45:52.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:45:52.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:45:52.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:45:52.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:45:52.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:45:52.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:45:52.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:45:52.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:45:52.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T11:45:52.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T11:45:52.097 INFO:tasks.workunit.client.0.trial128.stderr:start osd.9 2026-02-17T11:45:52.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:45:52.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-17T11:45:52.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T11:45:52.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:45:52.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:45:52.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:45:52.112 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:52.101+0000 7fca1be548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:52.114 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:52.105+0000 7fca1be548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:52.115 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:52.105+0000 7fca1be548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:45:52.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T11:45:52.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:45:52.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T11:45:52.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:45:52.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:45:52.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:52.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:45:52.249 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:45:52.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:52.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:45:52.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:53.355 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:53.345+0000 7fca1be548c0 -1 Falling back to public interface 2026-02-17T11:45:53.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:53.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:53.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:45:53.406 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:45:53.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:53.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:45:53.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:54.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:54.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:54.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:45:54.563 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:45:54.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:54.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:45:54.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:55.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:55.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:55.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:45:55.718 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:45:55.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:55.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:45:55.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:45:56.393 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:45:56.381+0000 7fca1be548c0 -1 osd.9 67 log_to_monitors true 2026-02-17T11:45:56.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:45:56.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:45:56.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:45:56.876 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:45:56.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:45:56.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:45:57.028 INFO:tasks.workunit.client.0.trial128.stderr:osd.9 up in weight 1 up_from 71 up_thru 0 down_at 68 last_clean_interval [44,67) [v2:127.0.0.1:6874/4158876231,v1:127.0.0.1:6875/4158876231] [v2:127.0.0.1:6876/4158876231,v1:127.0.0.1:6877/4158876231] exists,up e8ade1d3-5ef4-430b-bf60-7a9cf8ad1fa4 2026-02-17T11:45:57.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:45:57.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:45:57.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:45:57.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:45:57.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:45:57.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:45:57.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:45:57.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:45:57.029 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:45:57.029 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:45:57.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:45:57.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:45:57.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:45:57.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:45:57.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:45:57.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:45:57.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:45:57.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:45:57.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:45:57.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:45:57.291 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:45:57.291 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:45:57.291 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:45:57.291 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:45:57.291 INFO:tasks.workunit.client.0.trial128.stderr:5 2026-02-17T11:45:57.291 INFO:tasks.workunit.client.0.trial128.stderr:6 2026-02-17T11:45:57.291 INFO:tasks.workunit.client.0.trial128.stderr:7 2026-02-17T11:45:57.291 INFO:tasks.workunit.client.0.trial128.stderr:8 2026-02-17T11:45:57.291 INFO:tasks.workunit.client.0.trial128.stderr:9' 2026-02-17T11:45:57.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:45:57.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:57.292 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:45:57.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836507 2026-02-17T11:45:57.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836507 2026-02-17T11:45:57.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507' 2026-02-17T11:45:57.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:57.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:45:57.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705690 2026-02-17T11:45:57.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705690 2026-02-17T11:45:57.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-38654705690' 2026-02-17T11:45:57.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:57.435 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:45:57.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542168 2026-02-17T11:45:57.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542168 2026-02-17T11:45:57.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-60129542168' 2026-02-17T11:45:57.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:57.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:45:57.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874245 2026-02-17T11:45:57.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874245 2026-02-17T11:45:57.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-60129542168 3-279172874245' 2026-02-17T11:45:57.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:57.583 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:45:57.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247828 2026-02-17T11:45:57.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247828 2026-02-17T11:45:57.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-60129542168 3-279172874245 4-98784247828' 2026-02-17T11:45:57.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:57.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:45:57.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964117011 2026-02-17T11:45:57.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964117011 2026-02-17T11:45:57.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-60129542168 3-279172874245 4-98784247828 5-115964117011' 2026-02-17T11:45:57.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:57.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:45:57.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986193 2026-02-17T11:45:57.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986193 2026-02-17T11:45:57.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-60129542168 3-279172874245 4-98784247828 5-115964117011 6-133143986193' 2026-02-17T11:45:57.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:57.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:45:57.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855375 2026-02-17T11:45:57.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855375 2026-02-17T11:45:57.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-60129542168 3-279172874245 4-98784247828 5-115964117011 6-133143986193 7-150323855375' 2026-02-17T11:45:57.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:57.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:45:57.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724558 2026-02-17T11:45:57.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724558 2026-02-17T11:45:57.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-60129542168 3-279172874245 4-98784247828 5-115964117011 6-133143986193 7-150323855375 8-167503724558' 2026-02-17T11:45:57.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:45:57.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:45:58.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678018 2026-02-17T11:45:58.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678018 2026-02-17T11:45:58.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-60129542168 3-279172874245 4-98784247828 5-115964117011 6-133143986193 7-150323855375 8-167503724558 9-304942678018' 2026-02-17T11:45:58.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:58.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836507 2026-02-17T11:45:58.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:58.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:45:58.018 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836507 2026-02-17T11:45:58.019 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:58.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836507 2026-02-17T11:45:58.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836507' 2026-02-17T11:45:58.020 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836507 2026-02-17T11:45:58.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:45:58.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836506 -lt 21474836507 2026-02-17T11:45:58.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:45:59.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:45:59.177 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:45:59.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836508 -lt 21474836507 2026-02-17T11:45:59.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:59.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705690 2026-02-17T11:45:59.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:59.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:45:59.329 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705690 2026-02-17T11:45:59.329 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:59.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705690 2026-02-17T11:45:59.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705690' 2026-02-17T11:45:59.331 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705690 2026-02-17T11:45:59.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:45:59.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705690 -lt 38654705690 2026-02-17T11:45:59.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:59.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542168 2026-02-17T11:45:59.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:59.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:45:59.498 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542168 2026-02-17T11:45:59.498 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:59.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542168 2026-02-17T11:45:59.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542168' 2026-02-17T11:45:59.500 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542168 2026-02-17T11:45:59.500 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:45:59.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542168 -lt 60129542168 2026-02-17T11:45:59.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:59.663 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874245 2026-02-17T11:45:59.663 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:59.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:45:59.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874245 2026-02-17T11:45:59.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:59.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874245 2026-02-17T11:45:59.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874245' 2026-02-17T11:45:59.667 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.3 seq 279172874245 2026-02-17T11:45:59.667 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:45:59.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874246 -lt 279172874245 2026-02-17T11:45:59.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:59.818 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247828 2026-02-17T11:45:59.818 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:59.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:45:59.820 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247828 2026-02-17T11:45:59.820 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:59.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247828 2026-02-17T11:45:59.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247828' 2026-02-17T11:45:59.822 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.4 seq 98784247828 2026-02-17T11:45:59.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:45:59.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247829 -lt 98784247828 2026-02-17T11:45:59.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:45:59.980 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-115964117011 2026-02-17T11:45:59.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:45:59.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:45:59.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-115964117011 2026-02-17T11:45:59.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:45:59.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964117011 2026-02-17T11:45:59.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 115964117011' 2026-02-17T11:45:59.984 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.5 seq 115964117011 2026-02-17T11:45:59.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:46:00.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964117011 -lt 115964117011 2026-02-17T11:46:00.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:00.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986193 2026-02-17T11:46:00.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:00.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:46:00.145 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986193 2026-02-17T11:46:00.145 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:00.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986193 2026-02-17T11:46:00.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986193' 2026-02-17T11:46:00.146 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.6 seq 133143986193 2026-02-17T11:46:00.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:46:00.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986193 -lt 133143986193 2026-02-17T11:46:00.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:00.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-150323855375 2026-02-17T11:46:00.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:00.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:46:00.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-150323855375 2026-02-17T11:46:00.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:00.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855375 2026-02-17T11:46:00.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 150323855375' 2026-02-17T11:46:00.309 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.7 seq 150323855375 2026-02-17T11:46:00.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:46:00.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855375 -lt 150323855375 2026-02-17T11:46:00.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:00.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724558 2026-02-17T11:46:00.460 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:00.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:46:00.461 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724558 2026-02-17T11:46:00.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:00.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724558 2026-02-17T11:46:00.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724558' 2026-02-17T11:46:00.464 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.8 seq 167503724558 2026-02-17T11:46:00.464 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:46:00.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724558 -lt 167503724558 2026-02-17T11:46:00.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:00.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-304942678018 2026-02-17T11:46:00.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:00.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:46:00.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-304942678018 2026-02-17T11:46:00.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:00.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678018 2026-02-17T11:46:00.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 304942678018' 2026-02-17T11:46:00.625 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.9 seq 304942678018 2026-02-17T11:46:00.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:46:00.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678018 -lt 304942678018 2026-02-17T11:46:00.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:46:00.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:00.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:00.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:46:00.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:46:00.976 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:46:00.976 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:46:00.976 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:46:00.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:46:00.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:46:00.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:46:01.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:46:01.129 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:46:01.129 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:01.130 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:01.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:46:01.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:46:01.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:46:01.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:46:01.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:46:01.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:261: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 5 2026-02-17T11:46:01.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T11:46:01.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=ecpool 2026-02-17T11:46:01.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=5 2026-02-17T11:46:01.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:46:01.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:46:01.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:46:01.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T11:46:01.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:46:01.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:46:01.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:46:01.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:46:01.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T11:46:01.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:46:01.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T11:46:01.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:46:01.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:46:01.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:46:01.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:46:01.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:46:02.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:46:02.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:46:02.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:46:02.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:46:02.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T11:46:02.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:46:02.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:46:02.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-17T11:46:03.296 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T11:46:03.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T11:46:03.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:46:03.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:46:03.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T11:46:03.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:46:03.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:46:03.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:46:03.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:46:03.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:46:03.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:46:03.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T11:46:03.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T11:46:03.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:46:03.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:46:03.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:46:03.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:46:03.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:46:03.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:46:03.835 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:46:03.835 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:46:03.835 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:46:03.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:46:03.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:46:03.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:46:03.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:46:03.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:46:03.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:46:03.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:46:03.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:46:03.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:46:03.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:46:03.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:46:03.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T11:46:03.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T11:46:03.839 INFO:tasks.workunit.client.0.trial128.stderr:start osd.5 2026-02-17T11:46:03.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:46:03.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T11:46:03.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T11:46:03.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:46:03.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:46:03.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:46:03.857 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:46:03.845+0000 7f9d139c88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:03.859 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:46:03.849+0000 7f9d139c88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:03.859 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:46:03.849+0000 7f9d139c88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:03.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T11:46:03.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:46:03.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T11:46:03.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:46:03.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:46:03.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:03.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:46:03.987 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:46:03.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:03.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:46:04.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:05.107 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:46:05.097+0000 7f9d139c88c0 -1 Falling back to public interface 2026-02-17T11:46:05.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:05.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:05.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:46:05.141 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:46:05.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:05.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:46:05.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:06.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:06.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:06.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:46:06.294 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:46:06.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:06.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:46:06.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:07.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:07.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:07.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:46:07.451 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:46:07.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:07.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:46:07.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:08.074 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:46:08.061+0000 7f9d139c88c0 -1 osd.5 72 log_to_monitors true 2026-02-17T11:46:08.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:08.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:08.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:46:08.607 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:46:08.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:08.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:46:08.760 INFO:tasks.workunit.client.0.trial128.stderr:osd.5 up in weight 1 up_from 77 up_thru 77 down_at 73 last_clean_interval [27,72) [v2:127.0.0.1:6842/2297791011,v1:127.0.0.1:6843/2297791011] [v2:127.0.0.1:6844/2297791011,v1:127.0.0.1:6845/2297791011] exists,up 64f61c82-c6d6-4640-bd33-5fd8a48c3bfa 2026-02-17T11:46:08.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:46:08.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:46:08.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:46:08.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:46:08.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:46:08.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:46:08.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:46:08.761 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:46:08.762 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:46:08.762 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:46:08.762 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:46:08.762 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:46:08.762 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:46:08.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:46:08.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:46:08.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:46:08.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:46:08.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:46:08.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:46:09.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:46:09.025 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:46:09.025 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:46:09.025 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:46:09.026 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:46:09.026 INFO:tasks.workunit.client.0.trial128.stderr:5 2026-02-17T11:46:09.026 INFO:tasks.workunit.client.0.trial128.stderr:6 2026-02-17T11:46:09.026 INFO:tasks.workunit.client.0.trial128.stderr:7 2026-02-17T11:46:09.026 INFO:tasks.workunit.client.0.trial128.stderr:8 2026-02-17T11:46:09.026 INFO:tasks.workunit.client.0.trial128.stderr:9' 2026-02-17T11:46:09.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:46:09.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:09.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:46:09.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836511 2026-02-17T11:46:09.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836511 2026-02-17T11:46:09.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511' 2026-02-17T11:46:09.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:09.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:46:09.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705693 2026-02-17T11:46:09.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705693 2026-02-17T11:46:09.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705693' 2026-02-17T11:46:09.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:09.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:46:09.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542171 2026-02-17T11:46:09.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542171 2026-02-17T11:46:09.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705693 2-60129542171' 2026-02-17T11:46:09.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:09.241 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:46:09.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874249 2026-02-17T11:46:09.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874249 2026-02-17T11:46:09.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705693 2-60129542171 3-279172874249' 2026-02-17T11:46:09.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:09.311 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:46:09.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247832 2026-02-17T11:46:09.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247832 2026-02-17T11:46:09.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705693 2-60129542171 3-279172874249 4-98784247832' 2026-02-17T11:46:09.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:09.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:46:09.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=330712481794 2026-02-17T11:46:09.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 330712481794 2026-02-17T11:46:09.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705693 2-60129542171 3-279172874249 4-98784247832 5-330712481794' 2026-02-17T11:46:09.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:09.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:46:09.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986196 2026-02-17T11:46:09.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986196 2026-02-17T11:46:09.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705693 2-60129542171 3-279172874249 4-98784247832 5-330712481794 6-133143986196' 2026-02-17T11:46:09.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:09.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:46:09.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855378 2026-02-17T11:46:09.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855378 2026-02-17T11:46:09.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705693 2-60129542171 3-279172874249 4-98784247832 5-330712481794 6-133143986196 7-150323855378' 2026-02-17T11:46:09.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:09.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:46:09.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724562 2026-02-17T11:46:09.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724562 2026-02-17T11:46:09.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705693 2-60129542171 3-279172874249 4-98784247832 5-330712481794 6-133143986196 7-150323855378 8-167503724562' 2026-02-17T11:46:09.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:09.672 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:46:09.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678021 2026-02-17T11:46:09.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678021 2026-02-17T11:46:09.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705693 2-60129542171 3-279172874249 4-98784247832 5-330712481794 6-133143986196 7-150323855378 8-167503724562 9-304942678021' 2026-02-17T11:46:09.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:09.745 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836511 2026-02-17T11:46:09.745 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:09.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:46:09.747 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836511 2026-02-17T11:46:09.747 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:09.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836511 2026-02-17T11:46:09.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836511' 2026-02-17T11:46:09.749 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836511 2026-02-17T11:46:09.749 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:46:09.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836510 -lt 21474836511 2026-02-17T11:46:09.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:46:10.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:46:10.902 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:46:11.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836511 -lt 21474836511 2026-02-17T11:46:11.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:11.060 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705693 2026-02-17T11:46:11.060 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:11.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:46:11.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705693 2026-02-17T11:46:11.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:11.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705693 2026-02-17T11:46:11.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705693' 2026-02-17T11:46:11.064 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705693 2026-02-17T11:46:11.065 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:46:11.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705693 -lt 38654705693 2026-02-17T11:46:11.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:11.224 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542171 2026-02-17T11:46:11.224 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:11.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:46:11.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542171 2026-02-17T11:46:11.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:11.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542171 2026-02-17T11:46:11.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542171' 2026-02-17T11:46:11.227 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542171 2026-02-17T11:46:11.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:46:11.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542171 -lt 60129542171 2026-02-17T11:46:11.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:11.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874249 2026-02-17T11:46:11.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:11.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:46:11.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874249 2026-02-17T11:46:11.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:11.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874249 2026-02-17T11:46:11.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874249' 2026-02-17T11:46:11.389 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.3 seq 279172874249 2026-02-17T11:46:11.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:46:11.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874249 -lt 279172874249 2026-02-17T11:46:11.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:11.549 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247832 2026-02-17T11:46:11.549 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:11.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:46:11.551 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247832 2026-02-17T11:46:11.551 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:11.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247832 2026-02-17T11:46:11.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247832' 2026-02-17T11:46:11.553 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.4 seq 98784247832 2026-02-17T11:46:11.553 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:46:11.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247832 -lt 98784247832 2026-02-17T11:46:11.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:11.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-330712481794 2026-02-17T11:46:11.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:11.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:46:11.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-330712481794 2026-02-17T11:46:11.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:11.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=330712481794 2026-02-17T11:46:11.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 330712481794' 2026-02-17T11:46:11.711 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.5 seq 330712481794 2026-02-17T11:46:11.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:46:11.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 330712481794 -lt 330712481794 2026-02-17T11:46:11.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:11.876 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986196 2026-02-17T11:46:11.876 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:11.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:46:11.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986196 2026-02-17T11:46:11.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:11.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986196 2026-02-17T11:46:11.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986196' 2026-02-17T11:46:11.880 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.6 seq 133143986196 2026-02-17T11:46:11.880 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:46:12.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986196 -lt 133143986196 2026-02-17T11:46:12.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:12.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-150323855378 2026-02-17T11:46:12.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:12.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:46:12.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-150323855378 2026-02-17T11:46:12.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:12.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855378 2026-02-17T11:46:12.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 150323855378' 2026-02-17T11:46:12.042 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.7 seq 150323855378 2026-02-17T11:46:12.042 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:46:12.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855379 -lt 150323855378 2026-02-17T11:46:12.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:12.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724562 2026-02-17T11:46:12.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:12.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:46:12.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724562 2026-02-17T11:46:12.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:12.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724562 2026-02-17T11:46:12.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724562' 2026-02-17T11:46:12.203 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.8 seq 167503724562 2026-02-17T11:46:12.203 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:46:12.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724562 -lt 167503724562 2026-02-17T11:46:12.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:12.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-304942678021 2026-02-17T11:46:12.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:12.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:46:12.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-304942678021 2026-02-17T11:46:12.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:12.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678021 2026-02-17T11:46:12.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 304942678021' 2026-02-17T11:46:12.367 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.9 seq 304942678021 2026-02-17T11:46:12.367 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:46:12.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678021 -lt 304942678021 2026-02-17T11:46:12.531 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:46:12.531 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:12.531 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:12.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:46:12.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:46:12.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:46:12.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:46:12.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:46:12.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:46:12.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:46:12.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:46:12.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:46:12.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:46:12.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:12.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:13.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:46:13.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:46:13.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:46:13.054 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-17T11:46:13.054 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:46:13.054 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:46:13.054 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:46:13.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:46:13.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=2.0 2026-02-17T11:46:13.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 2.0 2026-02-17T11:46:13.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T11:46:13.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T11:46:13.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:46:13.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:46:13.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:46:13.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:46:13.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:45:48.306375+0000 2026-02-17T11:46:13.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T11:46:13.505 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T11:46:13.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T11:45:48.306375+0000 2026-02-17T11:46:13.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:46:13.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:45:48.306375+0000 2026-02-17T11:46:13.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:46:13.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:46:13.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:46:13.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:46:13.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:46:13.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:46:13.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:46:13.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:46:13.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:45:48.306375+0000 '>' 2026-02-17T11:45:48.306375+0000 2026-02-17T11:46:13.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:46:14.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:46:14.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:46:14.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:46:14.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:46:14.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:46:14.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:46:14.672 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:46:14.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:45:48.306375+0000 '>' 2026-02-17T11:45:48.306375+0000 2026-02-17T11:46:14.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:46:15.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:46:15.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:46:15.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:46:15.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:46:15.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:46:15.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:46:15.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:46:15.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:45:48.306375+0000 '>' 2026-02-17T11:45:48.306375+0000 2026-02-17T11:46:15.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:46:16.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:46:16.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:46:16.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:46:16.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:46:16.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:46:16.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:46:16.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:46:17.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:45:48.306375+0000 '>' 2026-02-17T11:45:48.306375+0000 2026-02-17T11:46:17.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:46:18.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:46:18.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:46:18.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:46:18.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:46:18.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:46:18.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:46:18.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:46:18.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:45:48.306375+0000 '>' 2026-02-17T11:45:48.306375+0000 2026-02-17T11:46:18.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:46:19.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:46:19.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:46:19.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:46:19.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:46:19.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:46:19.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:46:19.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:46:19.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:46:14.294017+0000 '>' 2026-02-17T11:45:48.306375+0000 2026-02-17T11:46:19.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:46:19.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:46:19.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:46:19.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:46:19.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T11:46:19.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:46:19.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:46:19.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:46:19.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:46:19.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T11:46:19.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:46:19.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-17T11:46:19.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:46:19.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:46:19.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:46:19.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:46:19.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:46:20.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:46:20.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:46:20.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:46:20.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:46:20.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T11:46:20.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:46:20.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:46:20.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-17T11:46:21.072 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:46:21.072 INFO:tasks.workunit.client.0.trial128.stdout:hinfo_key 2026-02-17T11:46:21.072 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:46:21.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-17T11:46:21.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:46:21.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:46:21.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T11:46:21.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:46:21.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:46:21.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:46:21.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:46:21.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:46:21.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:46:21.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T11:46:21.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T11:46:21.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:46:21.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:46:21.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:46:21.610 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:46:21.610 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:46:21.610 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:46:21.610 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:46:21.610 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:46:21.611 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:46:21.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:46:21.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:46:21.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:46:21.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:46:21.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:46:21.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:46:21.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:46:21.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:46:21.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:46:21.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:46:21.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:46:21.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T11:46:21.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T11:46:21.615 INFO:tasks.workunit.client.0.trial128.stderr:start osd.9 2026-02-17T11:46:21.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:46:21.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-17T11:46:21.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T11:46:21.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:46:21.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:46:21.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:46:21.629 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:46:21.616+0000 7fe197f808c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:21.630 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:46:21.620+0000 7fe197f808c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:21.631 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:46:21.620+0000 7fe197f808c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:21.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T11:46:21.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:46:21.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T11:46:21.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:46:21.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:46:21.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:21.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:46:21.769 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:46:21.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:21.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:46:21.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:22.888 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:46:22.876+0000 7fe197f808c0 -1 Falling back to public interface 2026-02-17T11:46:22.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:22.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:22.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:46:22.921 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:46:22.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:22.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:46:23.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:24.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:24.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:24.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:46:24.072 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:46:24.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:24.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:46:24.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:25.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:25.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:25.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:46:25.226 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:46:25.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:25.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:46:25.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:25.913 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:46:25.900+0000 7fe197f808c0 -1 osd.9 79 log_to_monitors true 2026-02-17T11:46:26.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:26.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:26.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:46:26.374 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:46:26.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:26.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:46:26.527 INFO:tasks.workunit.client.0.trial128.stderr:osd.9 up in weight 1 up_from 83 up_thru 0 down_at 80 last_clean_interval [71,79) [v2:127.0.0.1:6874/1290148448,v1:127.0.0.1:6875/1290148448] [v2:127.0.0.1:6876/1290148448,v1:127.0.0.1:6877/1290148448] exists,up e8ade1d3-5ef4-430b-bf60-7a9cf8ad1fa4 2026-02-17T11:46:26.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:46:26.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:46:26.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:46:26.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:46:26.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:46:26.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:46:26.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:46:26.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:46:26.529 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:46:26.529 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:46:26.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:46:26.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:46:26.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:46:26.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:46:26.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:46:26.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:46:26.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:46:26.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:46:26.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:46:26.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:46:26.784 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:46:26.784 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:46:26.784 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:46:26.784 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:46:26.784 INFO:tasks.workunit.client.0.trial128.stderr:5 2026-02-17T11:46:26.784 INFO:tasks.workunit.client.0.trial128.stderr:6 2026-02-17T11:46:26.784 INFO:tasks.workunit.client.0.trial128.stderr:7 2026-02-17T11:46:26.784 INFO:tasks.workunit.client.0.trial128.stderr:8 2026-02-17T11:46:26.785 INFO:tasks.workunit.client.0.trial128.stderr:9' 2026-02-17T11:46:26.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:46:26.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:26.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:46:26.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836515 2026-02-17T11:46:26.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836515 2026-02-17T11:46:26.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515' 2026-02-17T11:46:26.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:26.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:46:26.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705697 2026-02-17T11:46:26.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705697 2026-02-17T11:46:26.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697' 2026-02-17T11:46:26.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:26.929 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:46:26.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542176 2026-02-17T11:46:26.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542176 2026-02-17T11:46:26.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-60129542176' 2026-02-17T11:46:26.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:26.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:46:27.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874253 2026-02-17T11:46:27.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874253 2026-02-17T11:46:27.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-60129542176 3-279172874253' 2026-02-17T11:46:27.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:27.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:46:27.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247836 2026-02-17T11:46:27.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247836 2026-02-17T11:46:27.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-60129542176 3-279172874253 4-98784247836' 2026-02-17T11:46:27.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:27.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:46:27.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=330712481798 2026-02-17T11:46:27.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 330712481798 2026-02-17T11:46:27.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-60129542176 3-279172874253 4-98784247836 5-330712481798' 2026-02-17T11:46:27.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:27.217 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:46:27.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986201 2026-02-17T11:46:27.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986201 2026-02-17T11:46:27.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-60129542176 3-279172874253 4-98784247836 5-330712481798 6-133143986201' 2026-02-17T11:46:27.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:27.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:46:27.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855383 2026-02-17T11:46:27.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855383 2026-02-17T11:46:27.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-60129542176 3-279172874253 4-98784247836 5-330712481798 6-133143986201 7-150323855383' 2026-02-17T11:46:27.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:27.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:46:27.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724566 2026-02-17T11:46:27.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724566 2026-02-17T11:46:27.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-60129542176 3-279172874253 4-98784247836 5-330712481798 6-133143986201 7-150323855383 8-167503724566' 2026-02-17T11:46:27.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:27.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:46:27.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=356482285570 2026-02-17T11:46:27.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 356482285570 2026-02-17T11:46:27.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-60129542176 3-279172874253 4-98784247836 5-330712481798 6-133143986201 7-150323855383 8-167503724566 9-356482285570' 2026-02-17T11:46:27.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:27.511 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836515 2026-02-17T11:46:27.511 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:27.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:46:27.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836515 2026-02-17T11:46:27.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:27.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836515 2026-02-17T11:46:27.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836515' 2026-02-17T11:46:27.515 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836515 2026-02-17T11:46:27.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:46:27.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836515 -lt 21474836515 2026-02-17T11:46:27.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:27.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705697 2026-02-17T11:46:27.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:27.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:46:27.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705697 2026-02-17T11:46:27.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:27.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705697 2026-02-17T11:46:27.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705697' 2026-02-17T11:46:27.684 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705697 2026-02-17T11:46:27.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:46:27.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705696 -lt 38654705697 2026-02-17T11:46:27.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:46:28.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:46:28.849 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:46:29.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705698 -lt 38654705697 2026-02-17T11:46:29.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:29.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542176 2026-02-17T11:46:29.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:29.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:46:29.007 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542176 2026-02-17T11:46:29.007 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:29.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542176 2026-02-17T11:46:29.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542176' 2026-02-17T11:46:29.009 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542176 2026-02-17T11:46:29.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:46:29.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542176 -lt 60129542176 2026-02-17T11:46:29.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:29.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874253 2026-02-17T11:46:29.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:29.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:46:29.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874253 2026-02-17T11:46:29.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:29.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874253 2026-02-17T11:46:29.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874253' 2026-02-17T11:46:29.176 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.3 seq 279172874253 2026-02-17T11:46:29.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:46:29.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874254 -lt 279172874253 2026-02-17T11:46:29.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:29.333 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247836 2026-02-17T11:46:29.333 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:29.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:46:29.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247836 2026-02-17T11:46:29.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:29.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247836 2026-02-17T11:46:29.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247836' 2026-02-17T11:46:29.337 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.4 seq 98784247836 2026-02-17T11:46:29.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:46:29.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247837 -lt 98784247836 2026-02-17T11:46:29.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:29.499 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-330712481798 2026-02-17T11:46:29.500 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:29.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:46:29.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-330712481798 2026-02-17T11:46:29.502 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:29.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=330712481798 2026-02-17T11:46:29.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 330712481798' 2026-02-17T11:46:29.503 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.5 seq 330712481798 2026-02-17T11:46:29.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:46:29.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 330712481799 -lt 330712481798 2026-02-17T11:46:29.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:29.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986201 2026-02-17T11:46:29.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:29.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:46:29.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986201 2026-02-17T11:46:29.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:29.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986201 2026-02-17T11:46:29.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986201' 2026-02-17T11:46:29.671 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.6 seq 133143986201 2026-02-17T11:46:29.672 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:46:29.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986201 -lt 133143986201 2026-02-17T11:46:29.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:29.832 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-150323855383 2026-02-17T11:46:29.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:29.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:46:29.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-150323855383 2026-02-17T11:46:29.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:29.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855383 2026-02-17T11:46:29.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 150323855383' 2026-02-17T11:46:29.836 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.7 seq 150323855383 2026-02-17T11:46:29.837 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:46:29.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855383 -lt 150323855383 2026-02-17T11:46:29.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:29.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724566 2026-02-17T11:46:29.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:29.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:46:29.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724566 2026-02-17T11:46:29.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:30.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724566 2026-02-17T11:46:30.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724566' 2026-02-17T11:46:30.001 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.8 seq 167503724566 2026-02-17T11:46:30.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:46:30.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724566 -lt 167503724566 2026-02-17T11:46:30.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:30.166 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-356482285570 2026-02-17T11:46:30.166 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:30.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:46:30.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-356482285570 2026-02-17T11:46:30.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:30.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=356482285570 2026-02-17T11:46:30.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 356482285570' 2026-02-17T11:46:30.169 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.9 seq 356482285570 2026-02-17T11:46:30.170 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:46:30.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 356482285570 -lt 356482285570 2026-02-17T11:46:30.333 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:46:30.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:30.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:30.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:46:30.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:46:30.521 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:46:30.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:46:30.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:46:30.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:46:30.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:46:30.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:46:30.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:46:30.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:46:30.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:30.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:30.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:46:30.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:46:30.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:46:30.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:46:30.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:46:30.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:262: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 5 9 2026-02-17T11:46:30.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:185: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-17T11:46:30.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:186: corrupt_and_repair_two: local poolname=ecpool 2026-02-17T11:46:30.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: corrupt_and_repair_two: local first=5 2026-02-17T11:46:30.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: corrupt_and_repair_two: local second=9 2026-02-17T11:46:30.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:193: corrupt_and_repair_two: pids= 2026-02-17T11:46:30.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:46:30.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:46:30.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:46:30.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 194729"' 2026-02-17T11:46:30.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 194729' 2026-02-17T11:46:30.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 9 SOMETHING remove 2026-02-17T11:46:30.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:46:30.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:46:30.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 194731"' 2026-02-17T11:46:30.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 194731' 2026-02-17T11:46:30.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: corrupt_and_repair_two: wait_background pids 2026-02-17T11:46:30.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:46:30.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 194729 194731' 2026-02-17T11:46:30.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T11:46:30.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:46:30.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 194729 2026-02-17T11:46:30.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/194732: /' 2026-02-17T11:46:30.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 9 SOMETHING remove 2026-02-17T11:46:30.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/194734: /' 2026-02-17T11:46:33.384 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:46:33.384 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:46:33.384 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T11:46:33.384 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:46:33.384 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:46:33.384 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:46:33.384 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:46:33.384 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T11:46:33.384 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:46:33.384 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T11:46:33.384 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:46:33.384 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:46:33.385 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:46:33.388 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:46:33.389 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:46:33.389 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:46:33.389 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:46:33.389 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:46:33.389 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:46:33.389 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:46:33.389 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T11:46:33.389 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T11:46:33.389 INFO:tasks.workunit.client.0.trial128.stderr:194732: start osd.5 2026-02-17T11:46:33.389 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:46:33.622 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:46:33.623 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:46:33.623 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T11:46:33.623 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:46:33.623 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING remove 2026-02-17T11:46:33.623 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:46:33.623 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:46:33.623 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T11:46:33.623 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:46:33.623 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-17T11:46:33.623 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:46:33.623 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:46:33.623 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:46:33.624 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:46:33.624 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:46:33.624 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:46:33.624 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING remove 2026-02-17T11:46:33.624 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:46:33.624 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:46:33.624 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T11:46:33.624 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:46:33.624 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:46:33.624 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING remove 2026-02-17T11:46:33.624 INFO:tasks.workunit.client.0.trial128.stderr:194734: remove 2#2:eb822e21:::SOMETHING:head# 2026-02-17T11:46:33.624 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-17T11:46:33.624 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:46:33.624 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:46:33.624 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T11:46:33.624 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:46:33.625 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:46:33.625 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:46:33.625 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:46:33.626 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:46:33.626 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:46:33.626 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T11:46:33.626 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T11:46:33.626 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:46:33.626 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:46:33.626 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:46:33.626 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: start osd.9 2026-02-17T11:46:33.627 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:46:37.956 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T11:46:37.956 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T11:46:37.956 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:46:37.956 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:46:37.956 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:46:37.956 INFO:tasks.workunit.client.0.trial128.stderr:194732: 2026-02-17T11:46:33.400+0000 7f9fda4e48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:37.956 INFO:tasks.workunit.client.0.trial128.stderr:194732: 2026-02-17T11:46:33.400+0000 7f9fda4e48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:37.957 INFO:tasks.workunit.client.0.trial128.stderr:194732: 2026-02-17T11:46:33.400+0000 7f9fda4e48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:37.957 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T11:46:37.957 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:46:37.957 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T11:46:37.957 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:46:37.957 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:46:37.957 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:37.957 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:46:37.957 INFO:tasks.workunit.client.0.trial128.stderr:194732: 0 2026-02-17T11:46:37.957 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:37.957 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:46:37.957 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:37.958 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:37.958 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:37.958 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:46:37.958 INFO:tasks.workunit.client.0.trial128.stderr:194732: 1 2026-02-17T11:46:37.958 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:37.958 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:46:37.958 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:37.958 INFO:tasks.workunit.client.0.trial128.stderr:194732: 2026-02-17T11:46:34.944+0000 7f9fda4e48c0 -1 Falling back to public interface 2026-02-17T11:46:37.958 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:37.958 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:37.958 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:46:37.958 INFO:tasks.workunit.client.0.trial128.stderr:194732: 2 2026-02-17T11:46:37.958 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:37.959 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:46:37.959 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:37.959 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:37.959 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:37.959 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:46:37.959 INFO:tasks.workunit.client.0.trial128.stderr:194732: 3 2026-02-17T11:46:37.959 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:37.959 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:46:37.959 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:38.139 INFO:tasks.workunit.client.0.trial128.stderr:194732: 2026-02-17T11:46:37.944+0000 7f9fda4e48c0 194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-17T11:46:38.139 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T11:46:38.139 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:46:38.140 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:46:38.140 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:46:38.140 INFO:tasks.workunit.client.0.trial128.stderr:194734: 2026-02-17T11:46:33.628+0000 7f044c12e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:38.140 INFO:tasks.workunit.client.0.trial128.stderr:194734: 2026-02-17T11:46:33.628+0000 7f044c12e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:38.140 INFO:tasks.workunit.client.0.trial128.stderr:194734: 2026-02-17T11:46:33.632+0000 7f044c12e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:38.140 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T11:46:38.140 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:46:38.140 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T11:46:38.140 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:46:38.140 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:46:38.140 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:38.140 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:46:38.141 INFO:tasks.workunit.client.0.trial128.stderr:194734: 0 2026-02-17T11:46:38.141 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:38.141 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:46:38.141 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:38.141 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:38.141 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:38.141 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:46:38.141 INFO:tasks.workunit.client.0.trial128.stderr:194734: 1 2026-02-17T11:46:38.141 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:38.141 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:46:38.141 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:38.141 INFO:tasks.workunit.client.0.trial128.stderr:194734: 2026-02-17T11:46:35.096+0000 7f044c12e8c0 -1 Falling back to public interface 2026-02-17T11:46:38.142 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:38.142 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:38.142 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:46:38.142 INFO:tasks.workunit.client.0.trial128.stderr:194734: 2 2026-02-17T11:46:38.142 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:38.142 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:46:38.142 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:38.143 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:38.143 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:38.143 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:46:38.143 INFO:tasks.workunit.client.0.trial128.stderr:194734: 3 2026-02-17T11:46:38.143 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:38.143 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:46:38.143 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:38.651 INFO:tasks.workunit.client.0.trial128.stderr:194734: 2026-02-17T11:46:38.128+0000 7f044c12e8c0 -1 osd.5 84 log_to_monitors true 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: 4 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: osd.5 up in weight 1 up_from 89 up_thru 89 down_at 85 last_clean_interval [77,84) [v2:127.0.0.1:6842/2332430770,v1:127.0.0.1:6843/2332430770] [v2:127.0.0.1:6844/2332430770,v1:127.0.0.1:6845/2332430770] exists,up 64f61c82-c6d6-4640-bd33-5fd8a48c3bfa 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: 1 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: 2 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: 3 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: 4 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: 5 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: 6 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: 7 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: 8 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: 9' 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:46:38.652 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836519 2026-02-17T11:46:38.653 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836519 2026-02-17T11:46:38.653 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519' 2026-02-17T11:46:38.653 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:38.884 INFO:tasks.workunit.client.0.trial128.stderr:194732: -1 osd.9 84 log_to_monitors true 2026-02-17T11:46:38.884 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:38.884 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:38.884 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:46:38.884 INFO:tasks.workunit.client.0.trial128.stderr:194734: 4 2026-02-17T11:46:38.885 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:38.885 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:46:38.885 INFO:tasks.workunit.client.0.trial128.stderr:194734: osd.9 up in weight 1 up_from 91 up_thru 0 down_at 85 last_clean_interval [83,84) [v2:127.0.0.1:6874/2219749521,v1:127.0.0.1:6875/2219749521] [v2:127.0.0.1:6876/2219749521,v1:127.0.0.1:6877/2219749521] exists,up e8ade1d3-5ef4-430b-bf60-7a9cf8ad1fa4 2026-02-17T11:46:38.885 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:46:38.885 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:46:38.885 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:46:38.885 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:46:38.885 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:46:38.885 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:46:38.885 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:46:38.885 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:46:38.885 INFO:tasks.workunit.client.0.trial128.stderr:194734: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:46:38.886 INFO:tasks.workunit.client.0.trial128.stderr:194734: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:46:38.886 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:46:38.886 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:46:38.886 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:46:38.886 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:46:38.886 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:46:38.886 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:46:38.886 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:46:38.886 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:46:38.886 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:46:38.886 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:46:38.886 INFO:tasks.workunit.client.0.trial128.stderr:194734: 1 2026-02-17T11:46:38.887 INFO:tasks.workunit.client.0.trial128.stderr:194734: 2 2026-02-17T11:46:38.887 INFO:tasks.workunit.client.0.trial128.stderr:194734: 3 2026-02-17T11:46:38.887 INFO:tasks.workunit.client.0.trial128.stderr:194734: 4 2026-02-17T11:46:38.887 INFO:tasks.workunit.client.0.trial128.stderr:194734: 5 2026-02-17T11:46:38.887 INFO:tasks.workunit.client.0.trial128.stderr:194734: 6 2026-02-17T11:46:38.887 INFO:tasks.workunit.client.0.trial128.stderr:194734: 7 2026-02-17T11:46:38.887 INFO:tasks.workunit.client.0.trial128.stderr:194734: 8 2026-02-17T11:46:38.887 INFO:tasks.workunit.client.0.trial128.stderr:194734: 9' 2026-02-17T11:46:38.887 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:46:38.887 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:38.887 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:46:38.887 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836520 2026-02-17T11:46:38.888 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836520 2026-02-17T11:46:38.888 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520' 2026-02-17T11:46:38.888 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.162 INFO:tasks.workunit.client.0.trial128.stderr:194734: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:46:39.162 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705701 2026-02-17T11:46:39.162 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705701 2026-02-17T11:46:39.162 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701' 2026-02-17T11:46:39.162 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.162 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:46:39.162 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542179 2026-02-17T11:46:39.163 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542179 2026-02-17T11:46:39.163 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-60129542179' 2026-02-17T11:46:39.163 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.163 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:46:39.163 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874257 2026-02-17T11:46:39.163 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874257 2026-02-17T11:46:39.163 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-60129542179 3-279172874257' 2026-02-17T11:46:39.163 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.163 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:46:39.163 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247840 2026-02-17T11:46:39.163 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247840 2026-02-17T11:46:39.163 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-60129542179 3-279172874257 4-98784247840' 2026-02-17T11:46:39.164 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.164 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:46:39.164 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=382252089346 2026-02-17T11:46:39.164 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 382252089346 2026-02-17T11:46:39.164 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-60129542179 3-279172874257 4-98784247840 5-382252089346' 2026-02-17T11:46:39.164 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.164 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:46:39.164 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986204 2026-02-17T11:46:39.164 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986204 2026-02-17T11:46:39.164 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-60129542179 3-279172874257 4-98784247840 5-382252089346 6-133143986204' 2026-02-17T11:46:39.164 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.165 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:46:39.386 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_s/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:46:39.386 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705702 2026-02-17T11:46:39.386 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705702 2026-02-17T11:46:39.386 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702' 2026-02-17T11:46:39.386 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.386 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:46:39.386 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542180 2026-02-17T11:46:39.386 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542180 2026-02-17T11:46:39.387 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702 2-60129542180' 2026-02-17T11:46:39.387 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.387 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:46:39.387 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874258 2026-02-17T11:46:39.387 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874258 2026-02-17T11:46:39.387 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702 2-60129542180 3-279172874258' 2026-02-17T11:46:39.387 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.387 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:46:39.387 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247841 2026-02-17T11:46:39.387 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247841 2026-02-17T11:46:39.387 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702 2-60129542180 3-279172874258 4-98784247841' 2026-02-17T11:46:39.387 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.388 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:46:39.388 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=382252089347 2026-02-17T11:46:39.388 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 382252089347 2026-02-17T11:46:39.388 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702 2-60129542180 3-279172874258 4-98784247841 5-382252089347' 2026-02-17T11:46:39.388 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.388 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:46:39.388 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986205 2026-02-17T11:46:39.388 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986205 2026-02-17T11:46:39.388 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702 2-60129542180 3-279172874258 4-98784247841 5-382252089347 6-133143986205' 2026-02-17T11:46:39.388 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.388 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:46:39.471 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_sttats: seq=150323855386 2026-02-17T11:46:39.471 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855386 2026-02-17T11:46:39.471 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-60129542179 3-279172874257 4-98784247840 5-382252089346 6-133143986204 7-150323855386' 2026-02-17T11:46:39.472 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.472 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:46:39.472 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724570 2026-02-17T11:46:39.472 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724570 2026-02-17T11:46:39.472 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-60129542179 3-279172874257 4-98784247840 5-382252089346 6-133143986204 7-150323855386 8-167503724570' 2026-02-17T11:46:39.472 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.472 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:46:39.472 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=390842023938 2026-02-17T11:46:39.473 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 390842023938 2026-02-17T11:46:39.473 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-60129542179 3-279172874257 4-98784247840 5-382252089346 6-133143986204 7-150323855386 8-167503724570 9-390842023938' 2026-02-17T11:46:39.473 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:39.473 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836519 2026-02-17T11:46:39.473 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:39.473 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:46:39.473 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836519 2026-02-17T11:46:39.473 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:39.473 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836519 2026-02-17T11:46:39.473 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836519' 2026-02-17T11:46:39.473 INFO:tasks.workunit.client.0.trial128.stderr:194732: waiting osd.0 seq 21474836519 2026-02-17T11:46:39.473 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:46:39.474 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836520 -lt 21474836519 2026-02-17T11:46:39.474 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:39.474 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705701 2026-02-17T11:46:39.474 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:39.474 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:46:39.474 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705701 2026-02-17T11:46:39.474 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:39.474 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705701 2026-02-17T11:46:39.474 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705701' 2026-02-17T11:46:39.474 INFO:tasks.workunit.client.0.trial128.stderr:194732: waiting osd.1 seq 38654705701 2026-02-17T11:46:39.692 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ats: seq=150323855387 2026-02-17T11:46:39.692 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855387 2026-02-17T11:46:39.692 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702 2-60129542180 3-279172874258 4-98784247841 5-382252089347 6-133143986205 7-150323855387' 2026-02-17T11:46:39.692 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.692 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:46:39.692 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724571 2026-02-17T11:46:39.692 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724571 2026-02-17T11:46:39.692 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702 2-60129542180 3-279172874258 4-98784247841 5-382252089347 6-133143986205 7-150323855387 8-167503724571' 2026-02-17T11:46:39.693 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:39.693 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:46:39.693 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=390842023939 2026-02-17T11:46:39.693 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 390842023939 2026-02-17T11:46:39.693 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702 2-60129542180 3-279172874258 4-98784247841 5-382252089347 6-133143986205 7-150323855387 8-167503724571 9-390842023939' 2026-02-17T11:46:39.693 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:39.693 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836520 2026-02-17T11:46:39.693 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:39.693 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:46:39.693 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836520 2026-02-17T11:46:39.693 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:39.694 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836520 2026-02-17T11:46:39.694 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836520' 2026-02-17T11:46:39.694 INFO:tasks.workunit.client.0.trial128.stderr:194734: waiting osd.0 seq 21474836520 2026-02-17T11:46:39.694 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:46:39.694 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836520 -lt 21474836520 2026-02-17T11:46:39.694 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:39.694 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705702 2026-02-17T11:46:39.694 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:39.694 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:46:39.694 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705702 2026-02-17T11:46:39.694 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:39.694 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705702 2026-02-17T11:46:39.695 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705702' 2026-02-17T11:46:39.695 INFO:tasks.workunit.client.0.trial128.stderr:194734: waiting osd.1 seq 38654705702 2026-02-17T11:46:41.106 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: cceph osd last-stat-seq 1 2026-02-17T11:46:41.106 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705701 -lt 38654705701 2026-02-17T11:46:41.106 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:41.106 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542179 2026-02-17T11:46:41.107 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:41.107 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:46:41.107 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542179 2026-02-17T11:46:41.107 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:41.107 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542179 2026-02-17T11:46:41.107 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542179' 2026-02-17T11:46:41.107 INFO:tasks.workunit.client.0.trial128.stderr:194732: waiting osd.2 seq 60129542179 2026-02-17T11:46:41.107 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:46:41.107 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542179 -lt 60129542179 2026-02-17T11:46:41.107 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:41.107 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874257 2026-02-17T11:46:41.107 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:41.108 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:46:41.108 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874257 2026-02-17T11:46:41.108 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:41.108 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874257 2026-02-17T11:46:41.108 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874257' 2026-02-17T11:46:41.108 INFO:tasks.workunit.client.0.trial128.stderr:194732: waiting osd.3 seq 279172874257 2026-02-17T11:46:41.108 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:46:41.108 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874257 -lt 279172874257 2026-02-17T11:46:41.108 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:41.109 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247840 2026-02-17T11:46:41.109 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:41.109 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:46:41.109 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247840 2026-02-17T11:46:41.109 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:41.109 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247840 2026-02-17T11:46:41.109 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247840' 2026-02-17T11:46:41.109 INFO:tasks.workunit.client.0.trial128.stderr:194732: waiting osd.4 seq 98784247840 2026-02-17T11:46:41.109 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:46:41.109 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247839 -lt 98784247840 2026-02-17T11:46:41.109 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:46:41.109 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:46:41.328 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qaeph osd last-stat-seq 1 2026-02-17T11:46:41.329 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705701 -lt 38654705702 2026-02-17T11:46:41.329 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:46:41.329 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:46:41.329 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:46:41.329 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705702 -lt 38654705702 2026-02-17T11:46:41.329 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:41.329 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542180 2026-02-17T11:46:41.329 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:41.330 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:46:41.330 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542180 2026-02-17T11:46:41.330 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:41.330 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542180 2026-02-17T11:46:41.330 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542180' 2026-02-17T11:46:41.330 INFO:tasks.workunit.client.0.trial128.stderr:194734: waiting osd.2 seq 60129542180 2026-02-17T11:46:41.330 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:46:41.330 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542180 -lt 60129542180 2026-02-17T11:46:41.330 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:41.330 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874258 2026-02-17T11:46:41.330 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:41.330 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:46:41.330 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874258 2026-02-17T11:46:41.331 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:41.331 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874258 2026-02-17T11:46:41.331 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874258' 2026-02-17T11:46:41.331 INFO:tasks.workunit.client.0.trial128.stderr:194734: waiting osd.3 seq 279172874258 2026-02-17T11:46:41.331 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:46:41.331 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874258 -lt 279172874258 2026-02-17T11:46:41.331 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:41.331 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247841 2026-02-17T11:46:41.331 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:41.331 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:46:41.331 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247841 2026-02-17T11:46:41.331 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:41.332 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247841 2026-02-17T11:46:41.332 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247841' 2026-02-17T11:46:41.332 INFO:tasks.workunit.client.0.trial128.stderr:194734: waiting osd.4 seq 98784247841 2026-02-17T11:46:41.727 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa//standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:46:41.727 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247841 -lt 98784247840 2026-02-17T11:46:41.727 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:41.727 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-382252089346 2026-02-17T11:46:41.728 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:41.728 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:46:41.728 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-382252089346 2026-02-17T11:46:41.728 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:41.728 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=382252089346 2026-02-17T11:46:41.728 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 382252089346' 2026-02-17T11:46:41.728 INFO:tasks.workunit.client.0.trial128.stderr:194732: waiting osd.5 seq 382252089346 2026-02-17T11:46:41.728 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:46:41.728 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 382252089347 -lt 382252089346 2026-02-17T11:46:41.728 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:41.728 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986204 2026-02-17T11:46:41.729 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:41.729 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:46:41.729 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986204 2026-02-17T11:46:41.729 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:41.729 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986204 2026-02-17T11:46:41.729 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986204' 2026-02-17T11:46:41.729 INFO:tasks.workunit.client.0.trial128.stderr:194732: waiting osd.6 seq 133143986204 2026-02-17T11:46:41.729 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:46:41.729 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986205 -lt 133143986204 2026-02-17T11:46:41.729 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:41.729 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-150323855386 2026-02-17T11:46:41.729 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:41.729 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:46:41.730 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-150323855386 2026-02-17T11:46:41.730 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:41.730 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855386 2026-02-17T11:46:41.730 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 150323855386' 2026-02-17T11:46:41.730 INFO:tasks.workunit.client.0.trial128.stderr:194732: waiting osd.7 seq 150323855386 2026-02-17T11:46:41.730 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:46:41.730 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855388 -lt 150323855386 2026-02-17T11:46:41.730 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:41.991 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_sstandalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:46:41.991 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247841 -lt 98784247841 2026-02-17T11:46:41.991 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:41.991 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-382252089347 2026-02-17T11:46:41.991 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:41.991 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:46:41.992 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-382252089347 2026-02-17T11:46:41.992 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:41.992 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=382252089347 2026-02-17T11:46:41.992 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 382252089347' 2026-02-17T11:46:41.992 INFO:tasks.workunit.client.0.trial128.stderr:194734: waiting osd.5 seq 382252089347 2026-02-17T11:46:41.992 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:46:41.992 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 382252089347 -lt 382252089347 2026-02-17T11:46:41.992 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:41.992 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986205 2026-02-17T11:46:41.992 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:41.992 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:46:41.992 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986205 2026-02-17T11:46:41.993 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:41.993 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986205 2026-02-17T11:46:41.993 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986205' 2026-02-17T11:46:41.993 INFO:tasks.workunit.client.0.trial128.stderr:194734: waiting osd.6 seq 133143986205 2026-02-17T11:46:41.993 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:46:41.993 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986205 -lt 133143986205 2026-02-17T11:46:41.993 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:41.993 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-150323855387 2026-02-17T11:46:41.993 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:41.993 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:46:41.993 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-150323855387 2026-02-17T11:46:41.993 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:41.994 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855387 2026-02-17T11:46:41.994 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 150323855387' 2026-02-17T11:46:41.994 INFO:tasks.workunit.client.0.trial128.stderr:194734: waiting osd.7 seq 150323855387 2026-02-17T11:46:41.994 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:46:41.994 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855388 -lt 150323855387 2026-02-17T11:46:41.994 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:42.380 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_sttats: echo 8-167503724570 2026-02-17T11:46:42.380 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:42.380 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:46:42.380 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724570 2026-02-17T11:46:42.380 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:42.380 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724570 2026-02-17T11:46:42.380 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724570' 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: waiting osd.8 seq 167503724570 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724571 -lt 167503724570 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-390842023938 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-390842023938 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=390842023938 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 390842023938' 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: waiting osd.9 seq 390842023938 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 390842023939 -lt 390842023938 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:46:42.381 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:42.674 INFO:tasks.workunit.client.0.trial128.stderr:1947ats: echo 8-167503724571 2026-02-17T11:46:42.674 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:42.674 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:46:42.674 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724571 2026-02-17T11:46:42.674 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:42.674 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724571 2026-02-17T11:46:42.674 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724571' 2026-02-17T11:46:42.674 INFO:tasks.workunit.client.0.trial128.stderr:194734: waiting osd.8 seq 167503724571 2026-02-17T11:46:42.674 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:46:42.674 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724571 -lt 167503724571 2026-02-17T11:46:42.674 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:42.674 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-390842023939 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-390842023939 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=390842023939 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 390842023939' 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: waiting osd.9 seq 390842023939 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 390842023939 -lt 390842023939 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:46:42.675 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:43.207 INFO:tasks.workunit.client.0.trial128.stderr:1947332: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:43.208 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T11:46:43.208 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' -1 2026-02-17T11:46:43.208 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T11:46:43.208 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=3 2026-02-17T11:46:43.208 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:46:43.208 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T11:46:43.208 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:46:43.208 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:46:43.208 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:46:43.209 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:46:43.209 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:46:43.209 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:46:43.209 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:46:43.209 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:46:43.209 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T11:46:43.209 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:46:43.209 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:43.209 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:43.209 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T11:46:43.209 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T11:46:43.209 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:46:43.210 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:46:43.210 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:46:43.210 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:46:43.210 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:46:43.210 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:46:43.210 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:46:43.210 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:46:43.210 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:46:43.210 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:19474: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' -1 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=3 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:46:43.466 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:46:43.467 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:46:43.467 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:46:43.467 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:46:43.467 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:46:43.467 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:46:43.467 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:46:43.467 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T11:46:44.346 INFO:tasks.workunit.client.0.trial128.stderr:1947332: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:46:44.347 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T11:46:44.347 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:46:44.347 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:46:44.347 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:46:44.347 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:46:44.347 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:46:44.347 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:46:44.347 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:46:44.347 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:46:44.348 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T11:46:44.348 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:46:44.348 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:44.348 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:44.348 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T11:46:44.348 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T11:46:44.348 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:46:44.348 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:46:44.348 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:46:44.348 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:46:44.348 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:46:44.348 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:46:44.349 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:46:44.349 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:46:44.349 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:46:44.349 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T11:46:44.349 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:46:44.349 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T11:46:44.349 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:46:44.349 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:46:44.349 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:46:44.583 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/4: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:46:44.584 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T11:46:44.584 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:46:44.584 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:46:44.584 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:46:44.584 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:46:44.584 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:46:44.584 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:46:44.584 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:46:44.584 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:46:44.585 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T11:46:44.585 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:46:44.585 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:44.585 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:44.585 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T11:46:44.585 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T11:46:44.585 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:46:44.585 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:46:44.585 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:46:44.585 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:46:44.585 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:46:44.585 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:46:44.586 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:46:44.586 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:46:44.586 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:46:44.586 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T11:46:44.586 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:46:44.586 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T11:46:44.586 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:46:44.586 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:46:44.586 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:46:45.678 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/ccephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:46:45.678 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:46:45.678 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:46:45.679 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:46:45.679 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:46:45.679 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T11:46:45.679 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:46:45.679 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:45.679 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:45.679 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T11:46:45.679 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T11:46:45.679 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:46:45.679 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:46:45.679 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:46:45.680 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:46:45.680 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:46:45.680 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:46:45.680 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:46:45.680 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:46:45.680 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:46:45.680 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 13 )) 2026-02-17T11:46:45.680 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:46:45.680 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T11:46:45.680 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:46:45.680 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:46:45.680 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:46:45.681 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:46:45.681 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:46:45.681 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:46:45.681 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:46:45.916 INFO:tasks.workunit.client.0.trial128.stderr:ephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:46:45.916 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:46:45.916 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:46:45.916 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:46:45.916 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:46:45.916 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T11:46:45.916 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:46:45.916 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:45.917 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:45.917 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 5 2026-02-17T11:46:45.917 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 3 '!=' 3 2026-02-17T11:46:45.917 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T11:46:45.917 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T11:46:45.917 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T11:46:45.917 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T11:46:45.917 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T11:46:45.917 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T11:46:45.917 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T11:46:45.917 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T11:46:45.918 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T11:46:45.918 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 13 )) 2026-02-17T11:46:45.918 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T11:46:45.918 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T11:46:45.918 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T11:46:45.918 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:46:45.918 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:46:45.918 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:46:45.918 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:46:45.918 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:46:45.918 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:46:46.014 INFO:tasks.workunit.client.0.trial128.stderr:1194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:46:46.014 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:46:46.014 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:46:46.014 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:46.014 INFO:tasks.workunit.client.0.trial128.stderr:194732: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:46.014 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:46:46.014 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:46:46.014 INFO:tasks.workunit.client.0.trial128.stderr:194732: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:46:46.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:46:46.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:46:46.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 194731 2026-02-17T11:46:46.261 INFO:tasks.workunit.client.0.trial128.stderr:94734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:46:46.262 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:46:46.262 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:46:46.262 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:46:46.262 INFO:tasks.workunit.client.0.trial128.stderr:194734: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:46:46.262 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:46:46.262 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:46:46.262 INFO:tasks.workunit.client.0.trial128.stderr:194734: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:46:46.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:46:46.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T11:46:46.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T11:46:46.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T11:46:46.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: corrupt_and_repair_two: return_code=0 2026-02-17T11:46:46.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:198: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T11:46:46.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-17T11:46:46.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:46:46.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:46:46.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:46:46.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:46:46.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: local pg=2.0 2026-02-17T11:46:46.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:204: corrupt_and_repair_two: repair 2.0 2026-02-17T11:46:46.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T11:46:46.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T11:46:46.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:46:46.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:46:46.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:46:46.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:46:46.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:46:14.294017+0000 2026-02-17T11:46:46.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T11:46:46.714 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T11:46:46.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T11:46:14.294017+0000 2026-02-17T11:46:46.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:46:46.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:46:14.294017+0000 2026-02-17T11:46:46.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:46:46.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:46:46.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:46:46.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:46:46.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:46:46.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:46:46.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:46:46.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:46:46.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:46:14.294017+0000 '>' 2026-02-17T11:46:14.294017+0000 2026-02-17T11:46:46.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:46:47.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:46:47.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:46:47.880 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:46:47.880 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:46:47.880 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:46:47.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:46:47.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:46:48.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:46:14.294017+0000 '>' 2026-02-17T11:46:14.294017+0000 2026-02-17T11:46:48.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:46:49.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:46:49.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:46:49.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:46:49.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:46:49.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:46:49.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:46:49.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:46:49.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:46:47.114472+0000 '>' 2026-02-17T11:46:14.294017+0000 2026-02-17T11:46:49.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:46:49.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: corrupt_and_repair_two: pids= 2026-02-17T11:46:49.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T11:46:49.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:46:49.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:46:49.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 198771"' 2026-02-17T11:46:49.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 198771' 2026-02-17T11:46:49.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:46:49.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:46:49.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:46:49.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 198772"' 2026-02-17T11:46:49.188 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 198772' 2026-02-17T11:46:49.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:211: corrupt_and_repair_two: wait_background pids 2026-02-17T11:46:49.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 198771 198772' 2026-02-17T11:46:49.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T11:46:49.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:46:49.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 198771 2026-02-17T11:46:49.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T11:46:49.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/198774: /' 2026-02-17T11:46:49.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:46:49.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/198776: /' 2026-02-17T11:46:51.356 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:46:51.356 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:46:51.356 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T11:46:51.356 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:46:51.356 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T11:46:51.356 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:46:51.356 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:46:51.356 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T11:46:51.356 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:46:51.357 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T11:46:51.357 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:46:51.357 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:46:51.357 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:46:51.357 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:46:51.357 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:46:51.357 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:46:51.357 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T11:46:51.357 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:46:51.357 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:46:51.357 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T11:46:51.357 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:46:51.358 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:46:51.358 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING list-attrs 2026-02-17T11:46:51.358 INFO:tasks.workunit.client.0.trial128.stderr:198774: _ 2026-02-17T11:46:51.358 INFO:tasks.workunit.client.0.trial128.stderr:198774: hinfo_key 2026-02-17T11:46:51.358 INFO:tasks.workunit.client.0.trial128.stderr:198774: snapset 2026-02-17T11:46:51.358 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T11:46:51.358 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:46:51.358 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:46:51.358 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T11:46:51.358 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:46:51.358 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:46:51.358 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:46:51.359 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:46:51.359 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helper198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:46:51.359 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:46:51.359 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T11:46:51.359 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:46:51.359 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:46:51.359 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:46:51.359 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:46:51.360 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T11:46:51.360 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:46:51.360 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-17T11:46:51.360 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:46:51.360 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:46:51.360 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:46:51.360 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:46:51.360 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:46:51.360 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:46:51.360 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:46:51.360 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:46:51.360 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:46:51.361 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T11:46:51.361 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:46:51.361 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:46:51.361 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-17T11:46:51.361 INFO:tasks.workunit.client.0.trial128.stderr:198776: _ 2026-02-17T11:46:51.361 INFO:tasks.workunit.client.0.trial128.stderr:198776: hinfo_key 2026-02-17T11:46:51.361 INFO:tasks.workunit.client.0.trial128.stderr:198776: snapset 2026-02-17T11:46:51.361 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-17T11:46:51.361 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:46:51.361 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:46:51.361 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T11:46:51.361 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:46:51.362 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:46:51.362 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:46:51.362 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:46:51.362 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:46:51.363 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:46:51.363 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T11:46:51.363 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T11:46:51.363 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:46:51.363 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:46:51.363 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:46:51.363 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:46:51.363 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:46:51.363 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:46:51.363 INFO:tasks.workunit.client.0.trial128.stderr:198776: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:46:51.363 INFO:tasks.workunit.client.0.trial128.stderr:198776: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:46:51.364 INFO:tasks.workunit.client.0.trial128.stderr:198776: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:46:51.364 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:46:51.364 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:46:51.364 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:46:51.364 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:46:51.364 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:46:51.364 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:46:51.364 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:46:51.364 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:46:51.364 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:46:51.364 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:46:51.364 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:46:51.364 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T11:46:51.365 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T11:46:51.365 INFO:tasks.workunit.client.0.trial128.stderr:198776: start osd.9 2026-02-17T11:46:51.365 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recs.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:46:51.365 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:46:51.365 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T11:46:51.365 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T11:46:51.365 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:46:51.365 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:46:51.365 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:46:51.365 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:46:51.365 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:46:51.365 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:46:51.366 INFO:tasks.workunit.client.0.trial128.stderr:198774: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:46:51.366 INFO:tasks.workunit.client.0.trial128.stderr:198774: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:46:51.366 INFO:tasks.workunit.client.0.trial128.stderr:198774: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:46:51.366 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:46:51.366 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:46:51.366 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:46:51.366 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:46:51.366 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:46:51.366 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:46:51.366 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:46:51.366 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:46:51.366 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:46:51.366 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:46:51.367 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:46:51.367 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T11:46:51.367 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T11:46:51.367 INFO:tasks.workunit.client.0.trial128.stderr:198774: start osd.5 2026-02-17T11:46:55.700 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:46:55.700 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-17T11:46:55.701 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T11:46:55.701 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:46:55.701 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:46:55.701 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:46:55.701 INFO:tasks.workunit.client.0.trial128.stderr:198776: 2026-02-17T11:46:51.363+0000 7f3bfc9848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:55.701 INFO:tasks.workunit.client.0.trial128.stderr:198776: 2026-02-17T11:46:51.363+0000 7f3bfc9848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:55.701 INFO:tasks.workunit.client.0.trial128.stderr:198776: 2026-02-17T11:46:51.363+0000 7f3bfc9848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:55.701 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T11:46:55.701 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:46:55.701 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T11:46:55.701 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:46:55.702 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:46:55.702 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:55.702 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:46:55.702 INFO:tasks.workunit.client.0.trial128.stderr:198776: 0 2026-02-17T11:46:55.702 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:55.702 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:46:55.702 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:55.702 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:55.702 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:55.702 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:46:55.702 INFO:tasks.workunit.client.0.trial128.stderr:198776: 1 2026-02-17T11:46:55.703 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:55.703 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:46:55.703 INFO:tasks.workunit.client.0.trial128.stderr:198776: 2026-02-17T11:46:52.655+0000 7f3bfc9848c0 -1 Falling back to public interface 2026-02-17T11:46:55.703 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:55.703 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:55.703 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:55.703 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:46:55.703 INFO:tasks.workunit.client.0.trial128.stderr:198776: 2 2026-02-17T11:46:55.703 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:55.703 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:46:55.703 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:55.703 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:55.703 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:55.704 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:46:55.704 INFO:tasks.workunit.client.0.trial128.stderr:198776: 3 2026-02-17T11:46:55.704 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:55.704 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:46:55.704 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:55.712 INFO:tasks.workunit.client.0.trial128.stderr:198776: 2026-02-17T11:46:55.687+0000 7f3overy_ops 2026-02-17T11:46:55.713 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T11:46:55.713 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T11:46:55.713 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:46:55.713 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:46:55.713 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:46:55.713 INFO:tasks.workunit.client.0.trial128.stderr:198774: 2026-02-17T11:46:51.363+0000 7f64f34568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:55.713 INFO:tasks.workunit.client.0.trial128.stderr:198774: 2026-02-17T11:46:51.363+0000 7f64f34568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:55.713 INFO:tasks.workunit.client.0.trial128.stderr:198774: 2026-02-17T11:46:51.363+0000 7f64f34568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:46:55.713 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T11:46:55.713 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:46:55.713 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T11:46:55.714 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:46:55.714 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:46:55.714 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:55.714 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:46:55.714 INFO:tasks.workunit.client.0.trial128.stderr:198774: 0 2026-02-17T11:46:55.714 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:55.714 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:46:55.714 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:55.714 INFO:tasks.workunit.client.0.trial128.stderr:198774: 2026-02-17T11:46:52.643+0000 7f64f34568c0 -1 Falling back to public interface 2026-02-17T11:46:55.714 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:55.714 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:55.714 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:46:55.715 INFO:tasks.workunit.client.0.trial128.stderr:198774: 1 2026-02-17T11:46:55.715 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:55.715 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:46:55.715 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:55.715 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:55.715 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:55.715 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:46:55.715 INFO:tasks.workunit.client.0.trial128.stderr:198774: 2 2026-02-17T11:46:55.715 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:55.715 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:46:55.715 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:55.715 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:55.715 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:55.716 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:46:55.716 INFO:tasks.workunit.client.0.trial128.stderr:198774: 3 2026-02-17T11:46:55.716 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:55.716 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:46:55.716 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:46:56.632 INFO:tasks.workunit.client.0.trial128.stderr:198774: 2026-02-17T11:46:55.699+0000 7f6bfc9848c0 -1 osd.9 92 log_to_monitors true 2026-02-17T11:46:56.632 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:56.632 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:56.632 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:46:56.632 INFO:tasks.workunit.client.0.trial128.stderr:198776: 4 2026-02-17T11:46:56.632 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:56.633 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:46:56.633 INFO:tasks.workunit.client.0.trial128.stderr:198776: osd.9 up in weight 1 up_from 97 up_thru 0 down_at 93 last_clean_interval [91,92) [v2:127.0.0.1:6874/1209102429,v1:127.0.0.1:6875/1209102429] [v2:127.0.0.1:6876/1209102429,v1:127.0.0.1:6877/1209102429] exists,up e8ade1d3-5ef4-430b-bf60-7a9cf8ad1fa4 2026-02-17T11:46:56.633 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:46:56.633 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:46:56.633 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:46:56.633 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:46:56.633 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:46:56.633 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:46:56.633 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:46:56.633 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:46:56.633 INFO:tasks.workunit.client.0.trial128.stderr:198776: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:46:56.633 INFO:tasks.workunit.client.0.trial128.stderr:198776: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:46:56.634 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:46:56.634 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:46:56.634 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:46:56.634 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:46:56.634 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:46:56.634 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:46:56.634 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:46:56.634 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:46:56.634 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:46:56.634 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:46:56.634 INFO:tasks.workunit.client.0.trial128.stderr:198776: 1 2026-02-17T11:46:56.634 INFO:tasks.workunit.client.0.trial128.stderr:198776: 2 2026-02-17T11:46:56.635 INFO:tasks.workunit.client.0.trial128.stderr:198776: 3 2026-02-17T11:46:56.635 INFO:tasks.workunit.client.0.trial128.stderr:198776: 4 2026-02-17T11:46:56.635 INFO:tasks.workunit.client.0.trial128.stderr:198776: 5 2026-02-17T11:46:56.635 INFO:tasks.workunit.client.0.trial128.stderr:198776: 6 2026-02-17T11:46:56.635 INFO:tasks.workunit.client.0.trial128.stderr:198776: 7 2026-02-17T11:46:56.635 INFO:tasks.workunit.client.0.trial128.stderr:198776: 8 2026-02-17T11:46:56.635 INFO:tasks.workunit.client.0.trial128.stderr:198776: 9' 2026-02-17T11:46:56.635 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:46:56.635 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:56.635 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:46:56.635 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836524 2026-02-17T11:46:56.635 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836524 2026-02-17T11:46:56.636 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524' 2026-02-17T11:46:56.644 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids4f34568c0 -1 osd.5 92 log_to_monitors true 2026-02-17T11:46:56.644 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: 4 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: osd.5 up in weight 1 up_from 97 up_thru 97 down_at 93 last_clean_interval [89,92) [v2:127.0.0.1:6842/1033599512,v1:127.0.0.1:6843/1033599512] [v2:127.0.0.1:6844/1033599512,v1:127.0.0.1:6845/1033599512] exists,up 64f61c82-c6d6-4640-bd33-5fd8a48c3bfa 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: 1 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: 2 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: 3 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: 4 2026-02-17T11:46:56.645 INFO:tasks.workunit.client.0.trial128.stderr:198774: 5 2026-02-17T11:46:56.646 INFO:tasks.workunit.client.0.trial128.stderr:198774: 6 2026-02-17T11:46:56.646 INFO:tasks.workunit.client.0.trial128.stderr:198774: 7 2026-02-17T11:46:56.646 INFO:tasks.workunit.client.0.trial128.stderr:198774: 8 2026-02-17T11:46:56.646 INFO:tasks.workunit.client.0.trial128.stderr:198774: 9' 2026-02-17T11:46:56.646 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:46:56.646 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:56.646 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:46:56.646 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836525 2026-02-17T11:46:56.646 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836525 2026-02-17T11:46:56.646 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525' 2026-02-17T11:46:57.153 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.153 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:46:57.153 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705707 2026-02-17T11:46:57.153 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705707 2026-02-17T11:46:57.153 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707' 2026-02-17T11:46:57.153 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.154 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:46:57.154 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542186 2026-02-17T11:46:57.154 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542186 2026-02-17T11:46:57.154 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707 2-60129542186' 2026-02-17T11:46:57.154 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.154 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:46:57.154 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874262 2026-02-17T11:46:57.154 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874262 2026-02-17T11:46:57.154 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707 2-60129542186 3-279172874262' 2026-02-17T11:46:57.154 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.154 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:46:57.154 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247846 2026-02-17T11:46:57.155 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247846 2026-02-17T11:46:57.155 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707 2-60129542186 3-279172874262 4-98784247846' 2026-02-17T11:46:57.155 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.155 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:46:57.155 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=416611827714 2026-02-17T11:46:57.155 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 416611827714 2026-02-17T11:46:57.155 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707 2-60129542186 3-279172874262 4-98784247846 5-416611827714' 2026-02-17T11:46:57.155 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.155 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:46:57.155 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986210 2026-02-17T11:46:57.155 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986210 2026-02-17T11:46:57.156 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707 2-60129542186 3-279172874262 4-98784247846 5-416611827714 6-133143986210' 2026-02-17T11:46:57.156 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.156 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:46:57.156 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: 2026-02-17T11:46:57.156 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:46:57.157 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705706 2026-02-17T11:46:57.157 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705706 2026-02-17T11:46:57.157 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706' 2026-02-17T11:46:57.157 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.157 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:46:57.157 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542185 2026-02-17T11:46:57.157 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542185 2026-02-17T11:46:57.157 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-60129542185' 2026-02-17T11:46:57.157 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.157 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:46:57.157 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=279172874263 2026-02-17T11:46:57.158 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 279172874263 2026-02-17T11:46:57.158 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-60129542185 3-279172874263' 2026-02-17T11:46:57.158 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.158 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:46:57.158 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247845 2026-02-17T11:46:57.158 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247845 2026-02-17T11:46:57.158 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-60129542185 3-279172874263 4-98784247845' 2026-02-17T11:46:57.158 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.158 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:46:57.158 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=416611827715 2026-02-17T11:46:57.158 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 416611827715 2026-02-17T11:46:57.159 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-60129542185 3-279172874263 4-98784247845 5-416611827715' 2026-02-17T11:46:57.159 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.159 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:46:57.159 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986211 2026-02-17T11:46:57.159 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986211 2026-02-17T11:46:57.159 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-60129542185 3-279172874263 4-98784247845 5-416611827715 6-133143986211' 2026-02-17T11:46:57.159 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.159 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:46:57.465 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: fflush_pg_stats: seq=150323855392 2026-02-17T11:46:57.465 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855392 2026-02-17T11:46:57.465 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707 2-60129542186 3-279172874262 4-98784247846 5-416611827714 6-133143986210 7-150323855392' 2026-02-17T11:46:57.465 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.465 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:46:57.465 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724575 2026-02-17T11:46:57.465 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724575 2026-02-17T11:46:57.465 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707 2-60129542186 3-279172874262 4-98784247846 5-416611827714 6-133143986210 7-150323855392 8-167503724575' 2026-02-17T11:46:57.465 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.465 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:46:57.466 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=416611827714 2026-02-17T11:46:57.466 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 416611827714 2026-02-17T11:46:57.466 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707 2-60129542186 3-279172874262 4-98784247846 5-416611827714 6-133143986210 7-150323855392 8-167503724575 9-416611827714' 2026-02-17T11:46:57.466 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:57.466 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836525 2026-02-17T11:46:57.466 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:57.466 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:46:57.466 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836525 2026-02-17T11:46:57.467 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:57.467 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836525 2026-02-17T11:46:57.467 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836525' 2026-02-17T11:46:57.467 INFO:tasks.workunit.client.0.trial128.stderr:198774: waiting osd.0 seq 21474836525 2026-02-17T11:46:57.467 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:46:57.467 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836525 -lt 21474836525 2026-02-17T11:46:57.467 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:57.467 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705707 2026-02-17T11:46:57.467 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:57.467 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:46:57.467 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705707 2026-02-17T11:46:57.467 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:57.467 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705707 2026-02-17T11:46:57.468 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705707' 2026-02-17T11:46:57.468 INFO:tasks.workunit.client.0.trial128.stderr:198774: waiting osd.1 seq 38654705707 2026-02-17T11:46:57.468 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_plush_pg_stats: seq=150323855393 2026-02-17T11:46:57.468 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855393 2026-02-17T11:46:57.468 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-60129542185 3-279172874263 4-98784247845 5-416611827715 6-133143986211 7-150323855393' 2026-02-17T11:46:57.469 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.469 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:46:57.469 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724576 2026-02-17T11:46:57.469 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724576 2026-02-17T11:46:57.469 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-60129542185 3-279172874263 4-98784247845 5-416611827715 6-133143986211 7-150323855393 8-167503724576' 2026-02-17T11:46:57.469 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:46:57.469 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:46:57.469 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=416611827715 2026-02-17T11:46:57.469 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 416611827715 2026-02-17T11:46:57.469 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-60129542185 3-279172874263 4-98784247845 5-416611827715 6-133143986211 7-150323855393 8-167503724576 9-416611827715' 2026-02-17T11:46:57.469 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:57.469 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836524 2026-02-17T11:46:57.470 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:57.470 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:46:57.470 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836524 2026-02-17T11:46:57.470 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:57.470 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836524 2026-02-17T11:46:57.470 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836524' 2026-02-17T11:46:57.470 INFO:tasks.workunit.client.0.trial128.stderr:198776: waiting osd.0 seq 21474836524 2026-02-17T11:46:57.470 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:46:57.470 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836525 -lt 21474836524 2026-02-17T11:46:57.470 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:57.470 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705706 2026-02-17T11:46:57.470 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:57.471 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:46:57.471 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705706 2026-02-17T11:46:57.471 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:57.471 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705706 2026-02-17T11:46:57.471 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705706' 2026-02-17T11:46:57.471 INFO:tasks.workunit.client.0.trial128.stderr:198776: waiting osd.1 seq 38654705706 2026-02-17T11:46:59.082 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pgg_stats: ceph osd last-stat-seq 1 2026-02-17T11:46:59.082 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705707 -lt 38654705707 2026-02-17T11:46:59.082 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:59.082 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542186 2026-02-17T11:46:59.082 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:59.082 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542186 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542186 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542186' 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: waiting osd.2 seq 60129542186 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542186 -lt 60129542186 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874262 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874262 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874262 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874262' 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: waiting osd.3 seq 279172874262 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874263 -lt 279172874262 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247846 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247846 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247846 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247846' 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: waiting osd.4 seq 98784247846 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247844 -lt 98784247846 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:46:59.083 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.c_stats: ceph osd last-stat-seq 1 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705707 -lt 38654705706 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542185 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542185 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542185 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542185' 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: waiting osd.2 seq 60129542185 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542186 -lt 60129542185 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-279172874263 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-279172874263 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=279172874263 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 279172874263' 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: waiting osd.3 seq 279172874263 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 279172874263 -lt 279172874263 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247845 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247845 2026-02-17T11:46:59.114 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:59.115 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247845 2026-02-17T11:46:59.115 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247845' 2026-02-17T11:46:59.115 INFO:tasks.workunit.client.0.trial128.stderr:198776: waiting osd.4 seq 98784247845 2026-02-17T11:46:59.115 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:46:59.115 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247844 -lt 98784247845 2026-02-17T11:46:59.115 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:46:59.115 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:46:59.745 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.cllient.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:46:59.745 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247847 -lt 98784247846 2026-02-17T11:46:59.745 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:59.745 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-416611827714 2026-02-17T11:46:59.745 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:59.745 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:46:59.745 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-416611827714 2026-02-17T11:46:59.746 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:59.746 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=416611827714 2026-02-17T11:46:59.746 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 416611827714' 2026-02-17T11:46:59.746 INFO:tasks.workunit.client.0.trial128.stderr:198774: waiting osd.5 seq 416611827714 2026-02-17T11:46:59.746 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:46:59.746 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 416611827715 -lt 416611827714 2026-02-17T11:46:59.746 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:59.746 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986210 2026-02-17T11:46:59.746 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:59.746 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:46:59.746 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986210 2026-02-17T11:46:59.747 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:59.747 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986210 2026-02-17T11:46:59.747 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986210' 2026-02-17T11:46:59.747 INFO:tasks.workunit.client.0.trial128.stderr:198774: waiting osd.6 seq 133143986210 2026-02-17T11:46:59.747 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:46:59.747 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986211 -lt 133143986210 2026-02-17T11:46:59.747 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:59.747 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-150323855392 2026-02-17T11:46:59.747 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:59.747 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:46:59.747 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-150323855392 2026-02-17T11:46:59.747 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:59.748 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855392 2026-02-17T11:46:59.748 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 150323855392' 2026-02-17T11:46:59.748 INFO:tasks.workunit.client.0.trial128.stderr:198774: waiting osd.7 seq 150323855392 2026-02-17T11:46:59.748 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:46:59.748 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855393 -lt 150323855392 2026-02-17T11:46:59.748 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:59.749 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: ient.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:46:59.749 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247847 -lt 98784247845 2026-02-17T11:46:59.749 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:59.749 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-416611827715 2026-02-17T11:46:59.749 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:59.749 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:46:59.749 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-416611827715 2026-02-17T11:46:59.749 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:59.749 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=416611827715 2026-02-17T11:46:59.749 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 416611827715' 2026-02-17T11:46:59.749 INFO:tasks.workunit.client.0.trial128.stderr:198776: waiting osd.5 seq 416611827715 2026-02-17T11:46:59.749 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:46:59.750 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 416611827715 -lt 416611827715 2026-02-17T11:46:59.750 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:59.750 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986211 2026-02-17T11:46:59.750 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:59.750 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:46:59.750 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986211 2026-02-17T11:46:59.750 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:59.750 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986211 2026-02-17T11:46:59.750 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986211' 2026-02-17T11:46:59.750 INFO:tasks.workunit.client.0.trial128.stderr:198776: waiting osd.6 seq 133143986211 2026-02-17T11:46:59.750 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:46:59.750 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986211 -lt 133143986211 2026-02-17T11:46:59.751 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:46:59.751 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-150323855393 2026-02-17T11:46:59.751 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:46:59.751 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:46:59.751 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-150323855393 2026-02-17T11:46:59.751 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:46:59.751 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855393 2026-02-17T11:46:59.751 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 150323855393' 2026-02-17T11:46:59.751 INFO:tasks.workunit.client.0.trial128.stderr:198776: waiting osd.7 seq 150323855393 2026-02-17T11:46:59.751 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:46:59.751 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855393 -lt 150323855393 2026-02-17T11:46:59.751 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:00.394 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724576 2026-02-17T11:47:00.394 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:00.394 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:47:00.395 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724576 2026-02-17T11:47:00.395 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:00.395 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724576 2026-02-17T11:47:00.395 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724576' 2026-02-17T11:47:00.395 INFO:tasks.workunit.client.0.trial128.stderr:198776: waiting osd.8 seq 167503724576 2026-02-17T11:47:00.395 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:47:00.395 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724576 -lt 167503724576 2026-02-17T11:47:00.395 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:00.395 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-416611827715 2026-02-17T11:47:00.395 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:00.396 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:47:00.396 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-416611827715 2026-02-17T11:47:00.396 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:00.396 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=416611827715 2026-02-17T11:47:00.396 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 416611827715' 2026-02-17T11:47:00.396 INFO:tasks.workunit.client.0.trial128.stderr:198776: waiting osd.9 seq 416611827715 2026-02-17T11:47:00.396 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:47:00.396 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 416611827715 -lt 416611827715 2026-02-17T11:47:00.396 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:47:00.396 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:47:00.396 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:47:00.397 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:47:00.397 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:47:00.397 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:47:00.397 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:47:00.397 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:47:00.397 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:47:00.397 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:47:00.397 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:47:00.397 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:47:00.397 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:47:00.415 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json stflush_pg_stats: echo 8-167503724575 2026-02-17T11:47:00.416 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:00.416 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:47:00.416 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724575 2026-02-17T11:47:00.416 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:00.416 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724575 2026-02-17T11:47:00.416 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724575' 2026-02-17T11:47:00.416 INFO:tasks.workunit.client.0.trial128.stderr:198774: waiting osd.8 seq 167503724575 2026-02-17T11:47:00.416 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:47:00.416 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724576 -lt 167503724575 2026-02-17T11:47:00.416 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:00.416 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-416611827714 2026-02-17T11:47:00.417 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:00.417 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:47:00.417 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-416611827714 2026-02-17T11:47:00.417 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:00.417 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=416611827714 2026-02-17T11:47:00.417 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 416611827714' 2026-02-17T11:47:00.417 INFO:tasks.workunit.client.0.trial128.stderr:198774: waiting osd.9 seq 416611827714 2026-02-17T11:47:00.417 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:47:00.417 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 416611827715 -lt 416611827714 2026-02-17T11:47:00.417 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:47:00.417 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:47:00.417 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:47:00.418 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:47:00.418 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:47:00.418 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:47:00.418 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:47:00.418 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:47:00.418 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:47:00.418 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:47:00.418 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:47:00.418 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:47:00.418 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:47:00.577 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json satus 2026-02-17T11:47:00.578 INFO:tasks.workunit.client.0.trial128.stderr:198776: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:47:00.578 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:47:00.578 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:47:00.578 INFO:tasks.workunit.client.0.trial128.stderr:198776: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:47:00.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:47:00.604 INFO:tasks.workunit.client.0.trial128.stderr:tatus 2026-02-17T11:47:00.605 INFO:tasks.workunit.client.0.trial128.stderr:198774: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:47:00.605 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:47:00.605 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:47:00.605 INFO:tasks.workunit.client.0.trial128.stderr:198774: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:47:00.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:47:00.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:47:00.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 198772 2026-02-17T11:47:00.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T11:47:00.605 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T11:47:00.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T11:47:00.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:212: corrupt_and_repair_two: return_code=0 2026-02-17T11:47:00.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:213: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T11:47:00.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:47:00.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:47:00.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 3 5 2026-02-17T11:47:00.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:185: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-17T11:47:00.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:186: corrupt_and_repair_two: local poolname=ecpool 2026-02-17T11:47:00.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: corrupt_and_repair_two: local first=3 2026-02-17T11:47:00.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: corrupt_and_repair_two: local second=5 2026-02-17T11:47:00.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:193: corrupt_and_repair_two: pids= 2026-02-17T11:47:00.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:47:00.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:47:00.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:47:00.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 201788"' 2026-02-17T11:47:00.627 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 201788' 2026-02-17T11:47:00.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:47:00.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:47:00.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:47:00.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:47:00.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 201790"' 2026-02-17T11:47:00.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 201790' 2026-02-17T11:47:00.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: corrupt_and_repair_two: wait_background pids 2026-02-17T11:47:00.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 201788 201790' 2026-02-17T11:47:00.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T11:47:00.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:47:00.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 201788 2026-02-17T11:47:00.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/201791: /' 2026-02-17T11:47:00.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:47:00.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/201794: /' 2026-02-17T11:47:03.118 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:47:03.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:47:03.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T11:47:03.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:47:03.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:47:03.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:47:03.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:47:03.122 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:47:03.122 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:47:03.122 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T11:47:03.122 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T11:47:03.122 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:47:03.122 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:47:03.122 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: start osd.5 2026-02-17T11:47:03.123 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:47:03.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:47:03.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:47:03.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T11:47:03.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:47:03.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:47:03.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:47:03.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:47:03.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T11:47:03.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:47:03.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T11:47:03.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:47:03.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:47:03.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:47:03.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:47:03.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:47:03.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:47:03.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:47:03.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:47:03.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:47:03.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T11:47:03.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:47:03.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:47:03.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T11:47:03.144 INFO:tasks.workunit.client.0.trial128.stderr:201791: remove 0#2:eb822e21:::SOMETHING:head# 2026-02-17T11:47:03.144 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T11:47:03.144 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:47:03.144 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:47:03.144 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T11:47:03.144 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:47:03.144 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:47:03.144 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:47:03.144 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:47:03.145 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:47:03.145 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:47:03.145 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:47:03.145 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:47:03.145 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:47:03.146 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:47:03.146 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:47:03.146 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:47:03.146 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:47:03.146 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:47:03.146 INFO:tasks.workunit.client.0.trial128.stderr:201791: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:47:03.146 INFO:tasks.workunit.client.0.trial128.stderr:201791: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:47:03.146 INFO:tasks.workunit.client.0.trial128.stderr:201791: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:47:03.146 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:03.146 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:03.146 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:47:03.146 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:47:03.147 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:47:03.147 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:47:03.147 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:47:03.147 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:47:03.147 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:47:03.147 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:47:03.147 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:47:03.147 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:47:03.147 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T11:47:03.147 INFO:tasks.workunit.client.0.trial128.stderr:201791: start osd.3 2026-02-17T11:47:03.147 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:47:07.484 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: 2026-02-17T11:47:03.127+0000 7fbf9c2e78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: 2026-02-17T11:47:03.127+0000 7fbf9c2e78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: 2026-02-17T11:47:03.131+0000 7fbf9c2e78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: 0 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: 2026-02-17T11:47:04.411+0000 7fbf9c2e78c0 -1 Falling back to public interface 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: 1 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:47:07.485 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:07.486 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:07.486 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:07.486 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:47:07.486 INFO:tasks.workunit.client.0.trial128.stderr:201794: 2 2026-02-17T11:47:07.486 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:07.486 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:47:07.486 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:07.486 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:07.486 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:07.486 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:47:07.486 INFO:tasks.workunit.client.0.trial128.stderr:201794: 3 2026-02-17T11:47:07.486 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:07.486 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:47:07.486 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:07.736 INFO:tasks.workunit.client.0.trial128.stderr:201794: 2026-02-17T11:47:07.471+0000 7fbf9c2e78c0 201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T11:47:07.736 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T11:47:07.736 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:47:07.736 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:47:07.736 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:47:07.736 INFO:tasks.workunit.client.0.trial128.stderr:201791: 2026-02-17T11:47:03.151+0000 7fc46f79e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:07.736 INFO:tasks.workunit.client.0.trial128.stderr:201791: 2026-02-17T11:47:03.151+0000 7fc46f79e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:07.736 INFO:tasks.workunit.client.0.trial128.stderr:201791: 2026-02-17T11:47:03.155+0000 7fc46f79e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:07.736 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T11:47:07.736 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:47:07.736 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:47:07.736 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:47:07.736 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:47:07.736 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:07.736 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: 0 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: 1 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: 2026-02-17T11:47:04.691+0000 7fc46f79e8c0 -1 Falling back to public interface 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: 2 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: 3 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:47:07.737 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:08.394 INFO:tasks.workunit.client.0.trial128.stderr:201791: 2026-02-17T11:47:07.723+0000 7fc46f79e8c0 -1 osd.5 98 log_to_monitors true 2026-02-17T11:47:08.394 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:08.394 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:08.394 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:47:08.394 INFO:tasks.workunit.client.0.trial128.stderr:201794: 4 2026-02-17T11:47:08.394 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:08.394 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:47:08.394 INFO:tasks.workunit.client.0.trial128.stderr:201794: osd.5 up in weight 1 up_from 103 up_thru 103 down_at 99 last_clean_interval [97,98) [v2:127.0.0.1:6826/1677863249,v1:127.0.0.1:6827/1677863249] [v2:127.0.0.1:6828/1677863249,v1:127.0.0.1:6829/1677863249] exists,up 64f61c82-c6d6-4640-bd33-5fd8a48c3bfa 2026-02-17T11:47:08.394 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:47:08.394 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: 1 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: 2 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: 3 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: 4 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: 5 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: 6 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: 7 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: 8 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: 9' 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:47:08.395 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:08.396 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:47:08.396 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836528 2026-02-17T11:47:08.396 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836528 2026-02-17T11:47:08.396 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528' 2026-02-17T11:47:08.396 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:08.397 INFO:tasks.workunit.client.0.trial128.stderr:201794-1 osd.3 98 log_to_monitors true 2026-02-17T11:47:08.397 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:08.397 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: 4 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: osd.3 up in weight 1 up_from 105 up_thru 105 down_at 99 last_clean_interval [65,98) [v2:127.0.0.1:6842/2681587944,v1:127.0.0.1:6843/2681587944] [v2:127.0.0.1:6844/2681587944,v1:127.0.0.1:6845/2681587944] exists,up 3866ec67-d79b-45ae-83d0-612ac11a1b5b 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: 1 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: 2 2026-02-17T11:47:08.398 INFO:tasks.workunit.client.0.trial128.stderr:201791: 3 2026-02-17T11:47:08.399 INFO:tasks.workunit.client.0.trial128.stderr:201791: 4 2026-02-17T11:47:08.399 INFO:tasks.workunit.client.0.trial128.stderr:201791: 5 2026-02-17T11:47:08.399 INFO:tasks.workunit.client.0.trial128.stderr:201791: 6 2026-02-17T11:47:08.399 INFO:tasks.workunit.client.0.trial128.stderr:201791: 7 2026-02-17T11:47:08.399 INFO:tasks.workunit.client.0.trial128.stderr:201791: 8 2026-02-17T11:47:08.399 INFO:tasks.workunit.client.0.trial128.stderr:201791: 9' 2026-02-17T11:47:08.399 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:47:08.399 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:08.399 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:47:08.399 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836529 2026-02-17T11:47:08.399 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836529 2026-02-17T11:47:08.399 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529' 2026-02-17T11:47:08.399 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:08.897 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:47:08.897 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705712 2026-02-17T11:47:08.897 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705712 2026-02-17T11:47:08.898 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712' 2026-02-17T11:47:08.898 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:08.898 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:47:08.898 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542190 2026-02-17T11:47:08.898 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542190 2026-02-17T11:47:08.898 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712 2-60129542190' 2026-02-17T11:47:08.898 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:08.898 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:47:08.898 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=450971566082 2026-02-17T11:47:08.898 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 450971566082 2026-02-17T11:47:08.898 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712 2-60129542190 3-450971566082' 2026-02-17T11:47:08.898 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:08.899 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:47:08.899 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247850 2026-02-17T11:47:08.899 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247850 2026-02-17T11:47:08.899 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712 2-60129542190 3-450971566082 4-98784247850' 2026-02-17T11:47:08.899 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:08.899 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:47:08.899 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=442381631490 2026-02-17T11:47:08.899 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 442381631490 2026-02-17T11:47:08.899 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712 2-60129542190 3-450971566082 4-98784247850 5-442381631490' 2026-02-17T11:47:08.899 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:08.899 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:47:08.899 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986214 2026-02-17T11:47:08.900 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986214 2026-02-17T11:47:08.900 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712 2-60129542190 3-450971566082 4-98784247850 5-442381631490 6-133143986214' 2026-02-17T11:47:08.900 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:08.900 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:47:08.900 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:47:08.900 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705711 2026-02-17T11:47:08.900 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705711 2026-02-17T11:47:08.900 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711' 2026-02-17T11:47:08.900 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:08.900 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:47:08.900 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542189 2026-02-17T11:47:08.900 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542189 2026-02-17T11:47:08.901 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-60129542189' 2026-02-17T11:47:08.901 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:08.901 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:47:08.901 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=450971566083 2026-02-17T11:47:08.901 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 450971566083 2026-02-17T11:47:08.901 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-60129542189 3-450971566083' 2026-02-17T11:47:08.901 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:08.901 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:47:08.901 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247851 2026-02-17T11:47:08.901 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247851 2026-02-17T11:47:08.901 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-60129542189 3-450971566083 4-98784247851' 2026-02-17T11:47:08.901 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:08.901 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:47:08.902 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=442381631491 2026-02-17T11:47:08.902 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 442381631491 2026-02-17T11:47:08.902 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-60129542189 3-450971566083 4-98784247851 5-442381631491' 2026-02-17T11:47:08.902 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:08.902 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:47:08.902 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986215 2026-02-17T11:47:08.902 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986215 2026-02-17T11:47:08.902 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-60129542189 3-450971566083 4-98784247851 5-442381631491 6-133143986215' 2026-02-17T11:47:08.902 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:08.902 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:47:09.197 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855396 2026-02-17T11:47:09.198 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855396 2026-02-17T11:47:09.198 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712 2-60129542190 3-450971566082 4-98784247850 5-442381631490 6-133143986214 7-150323855396' 2026-02-17T11:47:09.198 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:09.198 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:47:09.198 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724579 2026-02-17T11:47:09.198 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724579 2026-02-17T11:47:09.198 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712 2-60129542190 3-450971566082 4-98784247850 5-442381631490 6-133143986214 7-150323855396 8-167503724579' 2026-02-17T11:47:09.198 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:09.198 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:47:09.198 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=416611827718 2026-02-17T11:47:09.199 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 416611827718 2026-02-17T11:47:09.199 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712 2-60129542190 3-450971566082 4-98784247850 5-442381631490 6-133143986214 7-150323855396 8-167503724579 9-416611827718' 2026-02-17T11:47:09.199 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:09.199 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836529 2026-02-17T11:47:09.199 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:09.199 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:47:09.199 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836529 2026-02-17T11:47:09.199 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:09.199 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836529 2026-02-17T11:47:09.199 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836529' 2026-02-17T11:47:09.199 INFO:tasks.workunit.client.0.trial128.stderr:201791: waiting osd.0 seq 21474836529 2026-02-17T11:47:09.200 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:47:09.200 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836530 -lt 21474836529 2026-02-17T11:47:09.200 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:09.200 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705712 2026-02-17T11:47:09.200 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:09.200 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:47:09.200 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705712 2026-02-17T11:47:09.200 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:09.200 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705712 2026-02-17T11:47:09.200 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705712' 2026-02-17T11:47:09.200 INFO:tasks.workunit.client.0.trial128.stderr:201791: waiting osd.1 seq 38654705712 2026-02-17T11:47:09.210 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats:_stats: seq=150323855397 2026-02-17T11:47:09.211 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855397 2026-02-17T11:47:09.211 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-60129542189 3-450971566083 4-98784247851 5-442381631491 6-133143986215 7-150323855397' 2026-02-17T11:47:09.211 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:09.211 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:47:09.211 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724580 2026-02-17T11:47:09.211 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724580 2026-02-17T11:47:09.211 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-60129542189 3-450971566083 4-98784247851 5-442381631491 6-133143986215 7-150323855397 8-167503724580' 2026-02-17T11:47:09.211 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:09.211 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:47:09.211 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=416611827719 2026-02-17T11:47:09.211 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 416611827719 2026-02-17T11:47:09.212 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-60129542189 3-450971566083 4-98784247851 5-442381631491 6-133143986215 7-150323855397 8-167503724580 9-416611827719' 2026-02-17T11:47:09.212 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:09.212 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836528 2026-02-17T11:47:09.212 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:09.212 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:47:09.212 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836528 2026-02-17T11:47:09.212 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:09.212 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836528 2026-02-17T11:47:09.212 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836528' 2026-02-17T11:47:09.212 INFO:tasks.workunit.client.0.trial128.stderr:201794: waiting osd.0 seq 21474836528 2026-02-17T11:47:09.212 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:47:09.212 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836530 -lt 21474836528 2026-02-17T11:47:09.213 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:09.213 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705711 2026-02-17T11:47:09.213 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:09.213 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:47:09.213 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705711 2026-02-17T11:47:09.213 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:09.213 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705711 2026-02-17T11:47:09.213 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705711' 2026-02-17T11:47:09.213 INFO:tasks.workunit.client.0.trial128.stderr:201794: waiting osd.1 seq 38654705711 2026-02-17T11:47:09.826 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:47:09.827 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705712 -lt 38654705712 2026-02-17T11:47:09.827 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:09.827 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542190 2026-02-17T11:47:09.827 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:09.827 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:47:09.828 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542190 2026-02-17T11:47:09.828 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:09.828 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542190 2026-02-17T11:47:09.828 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542190' 2026-02-17T11:47:09.828 INFO:tasks.workunit.client.0.trial128.stderr:201791: waiting osd.2 seq 60129542190 2026-02-17T11:47:09.828 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:47:09.829 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542190 -lt 60129542190 2026-02-17T11:47:09.829 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:09.829 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-450971566082 2026-02-17T11:47:09.829 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:09.829 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:47:09.829 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-450971566082 2026-02-17T11:47:09.829 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:09.829 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=450971566082 2026-02-17T11:47:09.830 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 450971566082' 2026-02-17T11:47:09.830 INFO:tasks.workunit.client.0.trial128.stderr:201791: waiting osd.3 seq 450971566082 2026-02-17T11:47:09.830 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:47:09.830 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566083 -lt 450971566082 2026-02-17T11:47:09.830 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:09.830 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247850 2026-02-17T11:47:09.830 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:09.830 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:47:09.830 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247850 2026-02-17T11:47:09.831 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:09.831 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247850 2026-02-17T11:47:09.831 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247850' 2026-02-17T11:47:09.831 INFO:tasks.workunit.client.0.trial128.stderr:201791: waiting osd.4 seq 98784247850 2026-02-17T11:47:09.831 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:47:09.831 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247851 -lt 98784247850 2026-02-17T11:47:09.831 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:09.831 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-442381631490 2026-02-17T11:47:09.860 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone ceph osd last-stat-seq 1 2026-02-17T11:47:09.860 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705712 -lt 38654705711 2026-02-17T11:47:09.860 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:09.860 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542189 2026-02-17T11:47:09.860 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:09.860 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:47:09.860 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542189 2026-02-17T11:47:09.860 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:09.860 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542189 2026-02-17T11:47:09.861 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542189' 2026-02-17T11:47:09.861 INFO:tasks.workunit.client.0.trial128.stderr:201794: waiting osd.2 seq 60129542189 2026-02-17T11:47:09.861 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:47:09.861 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542190 -lt 60129542189 2026-02-17T11:47:09.861 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:09.861 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-450971566083 2026-02-17T11:47:09.861 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:09.861 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:47:09.861 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-450971566083 2026-02-17T11:47:09.861 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:09.861 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=450971566083 2026-02-17T11:47:09.861 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 450971566083' 2026-02-17T11:47:09.862 INFO:tasks.workunit.client.0.trial128.stderr:201794: waiting osd.3 seq 450971566083 2026-02-17T11:47:09.862 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:47:09.862 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566083 -lt 450971566083 2026-02-17T11:47:09.862 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:09.862 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247851 2026-02-17T11:47:09.862 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:09.862 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:47:09.862 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247851 2026-02-17T11:47:09.862 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:09.862 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247851 2026-02-17T11:47:09.862 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247851' 2026-02-17T11:47:09.862 INFO:tasks.workunit.client.0.trial128.stderr:201794: waiting osd.4 seq 98784247851 2026-02-17T11:47:09.863 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:47:09.863 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247851 -lt 98784247851 2026-02-17T11:47:09.863 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:09.863 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-442381631491 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-442381631490 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=442381631490 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 442381631490' 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: waiting osd.5 seq 442381631490 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 442381631491 -lt 442381631490 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986214 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986214 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986214 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986214' 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: waiting osd.6 seq 133143986214 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986215 -lt 133143986214 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:10.309 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-150323855396 2026-02-17T11:47:10.310 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:10.310 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:47:10.310 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-150323855396 2026-02-17T11:47:10.310 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:10.310 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855396 2026-02-17T11:47:10.310 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 150323855396' 2026-02-17T11:47:10.310 INFO:tasks.workunit.client.0.trial128.stderr:201791: waiting osd.7 seq 150323855396 2026-02-17T11:47:10.310 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:47:10.310 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855397 -lt 150323855396 2026-02-17T11:47:10.310 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:10.310 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724579 2026-02-17T11:47:10.310 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:10.310 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:47:10.310 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724579 2026-02-17T11:47:10.319 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:10.319 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:47:10.319 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-442381631491 2026-02-17T11:47:10.319 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:10.319 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=442381631491 2026-02-17T11:47:10.319 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 442381631491' 2026-02-17T11:47:10.319 INFO:tasks.workunit.client.0.trial128.stderr:201794: waiting osd.5 seq 442381631491 2026-02-17T11:47:10.319 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:47:10.319 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 442381631491 -lt 442381631491 2026-02-17T11:47:10.319 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986215 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986215 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986215 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986215' 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: waiting osd.6 seq 133143986215 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986215 -lt 133143986215 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-150323855397 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-150323855397 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855397 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 150323855397' 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: waiting osd.7 seq 150323855397 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855397 -lt 150323855397 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724580 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:47:10.320 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724580 2026-02-17T11:47:12.118 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:12.118 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724580 2026-02-17T11:47:12.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724580' 2026-02-17T11:47:12.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: waiting osd.8 seq 167503724580 2026-02-17T11:47:12.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:47:12.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724578 -lt 167503724580 2026-02-17T11:47:12.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:47:12.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:47:12.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:47:12.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724581 -lt 167503724580 2026-02-17T11:47:12.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:12.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-416611827719 2026-02-17T11:47:12.119 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:12.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:47:12.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-416611827719 2026-02-17T11:47:12.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:12.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=416611827719 2026-02-17T11:47:12.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 416611827719' 2026-02-17T11:47:12.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: waiting osd.9 seq 416611827719 2026-02-17T11:47:12.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:47:12.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 416611827720 -lt 416611827719 2026-02-17T11:47:12.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:47:12.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:47:12.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:47:12.120 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:47:12.121 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:47:12.121 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:47:12.121 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:47:12.121 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:47:12.121 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:47:12.121 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:47:12.121 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:47:12.121 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:47:12.121 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:47:12.121 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:47:12.140 INFO:tasks.workunit.client.0.trial128.stderr: 2026-02-17T11:47:12.140 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724579 2026-02-17T11:47:12.141 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724579' 2026-02-17T11:47:12.141 INFO:tasks.workunit.client.0.trial128.stderr:201791: waiting osd.8 seq 167503724579 2026-02-17T11:47:12.141 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:47:12.141 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724578 -lt 167503724579 2026-02-17T11:47:12.141 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:47:12.141 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:47:12.141 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:47:12.141 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724581 -lt 167503724579 2026-02-17T11:47:12.141 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:12.141 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-416611827718 2026-02-17T11:47:12.141 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:12.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:47:12.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-416611827718 2026-02-17T11:47:12.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:12.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=416611827718 2026-02-17T11:47:12.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 416611827718' 2026-02-17T11:47:12.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: waiting osd.9 seq 416611827718 2026-02-17T11:47:12.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:47:12.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 416611827720 -lt 416611827718 2026-02-17T11:47:12.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:47:12.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:47:12.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:47:12.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:47:12.142 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:47:12.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:47:12.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:47:12.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:47:12.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:47:12.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:47:12.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:47:12.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:47:12.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:47:12.143 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:47:12.296 INFO:tasks.workunit.client.0.trial128.stderr:201794: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:47:12.296 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:47:12.296 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:47:12.297 INFO:tasks.workunit.client.0.trial128.stderr:201794: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:47:12.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:47:12.329 INFO:tasks.workunit.client.0.trial128.stderr:201791: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:47:12.330 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:47:12.330 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:47:12.330 INFO:tasks.workunit.client.0.trial128.stderr:201791: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:47:12.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:47:12.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:47:12.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 201790 2026-02-17T11:47:12.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T11:47:12.330 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T11:47:12.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T11:47:12.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: corrupt_and_repair_two: return_code=0 2026-02-17T11:47:12.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:198: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T11:47:12.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-17T11:47:12.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:47:12.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:47:12.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:47:12.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:47:12.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: local pg=2.0 2026-02-17T11:47:12.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:204: corrupt_and_repair_two: repair 2.0 2026-02-17T11:47:12.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T11:47:12.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T11:47:12.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:47:12.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:47:12.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:47:12.480 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:47:12.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:46:47.114472+0000 2026-02-17T11:47:12.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T11:47:12.776 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T11:47:12.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T11:46:47.114472+0000 2026-02-17T11:47:12.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:47:12.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:46:47.114472+0000 2026-02-17T11:47:12.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:47:12.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:47:12.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:47:12.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:47:12.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:47:12.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:47:12.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:47:12.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:47:12.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:46:47.114472+0000 '>' 2026-02-17T11:46:47.114472+0000 2026-02-17T11:47:12.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:47:13.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:47:13.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:47:13.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:47:13.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:47:13.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:47:13.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:47:13.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:47:14.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:46:47.114472+0000 '>' 2026-02-17T11:46:47.114472+0000 2026-02-17T11:47:14.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:47:15.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:47:15.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:47:15.085 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:47:15.085 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:47:15.085 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:47:15.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:47:15.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:47:15.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:46:47.114472+0000 '>' 2026-02-17T11:46:47.114472+0000 2026-02-17T11:47:15.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:47:16.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:47:16.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:47:16.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:47:16.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:47:16.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:47:16.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:47:16.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:47:16.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:46:47.114472+0000 '>' 2026-02-17T11:46:47.114472+0000 2026-02-17T11:47:16.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:47:17.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:47:17.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:47:17.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:47:17.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:47:17.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:47:17.396 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:47:17.396 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:47:17.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:46:47.114472+0000 '>' 2026-02-17T11:46:47.114472+0000 2026-02-17T11:47:17.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:47:18.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:47:18.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:47:18.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:47:18.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:47:18.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:47:18.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:47:18.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:47:18.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:46:47.114472+0000 '>' 2026-02-17T11:46:47.114472+0000 2026-02-17T11:47:18.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:47:19.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:47:19.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:47:19.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:47:19.714 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:47:19.714 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:47:19.714 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:47:19.714 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:47:19.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:47:13.635263+0000 '>' 2026-02-17T11:46:47.114472+0000 2026-02-17T11:47:19.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:47:19.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: corrupt_and_repair_two: pids= 2026-02-17T11:47:19.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 205395"' 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 205395' 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 205396"' 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 205396' 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:211: corrupt_and_repair_two: wait_background pids 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 205395 205396' 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 205395 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/205398: /' 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T11:47:19.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/205400: /' 2026-02-17T11:47:22.031 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:47:22.031 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:47:22.031 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T11:47:22.031 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:47:22.031 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:47:22.031 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:47:22.032 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:47:22.032 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T11:47:22.032 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:47:22.032 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T11:47:22.032 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:47:22.032 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:47:22.032 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:47:22.032 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:47:22.032 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:47:22.032 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:47:22.032 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:47:22.033 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:47:22.033 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:47:22.033 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T11:47:22.033 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:47:22.033 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:47:22.033 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-17T11:47:22.033 INFO:tasks.workunit.client.0.trial128.stderr:205398: Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-17T11:47:22.033 INFO:tasks.workunit.client.0.trial128.stderr:205398: _ 2026-02-17T11:47:22.033 INFO:tasks.workunit.client.0.trial128.stderr:205398: hinfo_key 2026-02-17T11:47:22.033 INFO:tasks.workunit.client.0.trial128.stderr:205398: snapset 2026-02-17T11:47:22.033 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T11:47:22.034 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:47:22.034 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:47:22.034 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T11:47:22.034 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:47:22.034 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:47:22.034 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:47:22.034 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' 205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:47:22.034 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:47:22.034 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T11:47:22.034 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:47:22.035 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T11:47:22.035 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:47:22.035 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:47:22.035 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T11:47:22.035 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:47:22.035 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T11:47:22.035 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:47:22.035 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:47:22.035 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:47:22.036 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:47:22.036 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:47:22.036 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:47:22.036 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T11:47:22.036 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:47:22.036 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:47:22.036 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T11:47:22.036 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:47:22.036 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:47:22.036 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING list-attrs 2026-02-17T11:47:22.036 INFO:tasks.workunit.client.0.trial128.stderr:205400: _ 2026-02-17T11:47:22.036 INFO:tasks.workunit.client.0.trial128.stderr:205400: hinfo_key 2026-02-17T11:47:22.037 INFO:tasks.workunit.client.0.trial128.stderr:205400: snapset 2026-02-17T11:47:22.037 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T11:47:22.037 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:47:22.037 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:47:22.037 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T11:47:22.037 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:47:22.037 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:47:22.037 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:47:22.037 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:47:22.038 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helper--osd-failsafe-full-ratio=.99' 2026-02-17T11:47:22.038 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:47:22.038 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:47:22.038 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:47:22.039 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:47:22.039 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:47:22.039 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:47:22.039 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:47:22.039 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:47:22.039 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:47:22.039 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:47:22.039 INFO:tasks.workunit.client.0.trial128.stderr:205398: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:47:22.039 INFO:tasks.workunit.client.0.trial128.stderr:205398: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:47:22.039 INFO:tasks.workunit.client.0.trial128.stderr:205398: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:47:22.039 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:22.039 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:22.040 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:47:22.040 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:47:22.040 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:47:22.040 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:47:22.040 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:47:22.040 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:47:22.040 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:47:22.040 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:47:22.040 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:47:22.040 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:47:22.040 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T11:47:22.041 INFO:tasks.workunit.client.0.trial128.stderr:205398: start osd.3 2026-02-17T11:47:22.041 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-s.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:47:22.041 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:47:22.041 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T11:47:22.041 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T11:47:22.041 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:47:22.041 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:47:22.041 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:47:22.041 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:47:22.041 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:47:22.042 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:47:22.042 INFO:tasks.workunit.client.0.trial128.stderr:205400: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:47:22.042 INFO:tasks.workunit.client.0.trial128.stderr:205400: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:47:22.042 INFO:tasks.workunit.client.0.trial128.stderr:205400: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:47:22.042 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:22.042 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:22.042 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:47:22.042 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:47:22.043 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:47:22.043 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:47:22.043 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:47:22.043 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:47:22.043 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:47:22.043 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:47:22.043 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:47:22.043 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T11:47:22.043 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T11:47:22.044 INFO:tasks.workunit.client.0.trial128.stderr:205400: start osd.5 2026-02-17T11:47:25.803 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_reclen=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:47:25.804 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T11:47:25.804 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T11:47:25.804 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:47:25.804 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:47:25.804 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:47:25.804 INFO:tasks.workunit.client.0.trial128.stderr:205398: 2026-02-17T11:47:22.034+0000 7f0ba63708c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:25.804 INFO:tasks.workunit.client.0.trial128.stderr:205398: 2026-02-17T11:47:22.034+0000 7f0ba63708c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:25.804 INFO:tasks.workunit.client.0.trial128.stderr:205398: 2026-02-17T11:47:22.038+0000 7f0ba63708c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:25.805 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T11:47:25.805 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:47:25.805 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:47:25.805 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:47:25.805 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:47:25.805 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:25.805 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:47:25.805 INFO:tasks.workunit.client.0.trial128.stderr:205398: 0 2026-02-17T11:47:25.805 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:25.805 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:47:25.805 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:25.805 INFO:tasks.workunit.client.0.trial128.stderr:205398: 2026-02-17T11:47:23.318+0000 7f0ba63708c0 -1 Falling back to public interface 2026-02-17T11:47:25.806 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:25.806 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:25.806 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:47:25.806 INFO:tasks.workunit.client.0.trial128.stderr:205398: 1 2026-02-17T11:47:25.806 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:25.806 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:47:25.806 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:25.806 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:25.806 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:25.806 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:47:25.806 INFO:tasks.workunit.client.0.trial128.stderr:205398: 2 2026-02-17T11:47:25.806 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:25.807 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:47:25.807 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:25.807 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:25.807 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:25.807 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:47:25.807 INFO:tasks.workunit.client.0.trial128.stderr:205398: 3 2026-02-17T11:47:25.807 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:25.807 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:47:26.651 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.clientovery_ops 2026-02-17T11:47:26.651 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T11:47:26.651 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T11:47:26.651 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:47:26.651 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:47:26.651 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:47:26.651 INFO:tasks.workunit.client.0.trial128.stderr:205400: 2026-02-17T11:47:22.034+0000 7f22bc8118c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:26.651 INFO:tasks.workunit.client.0.trial128.stderr:205400: 2026-02-17T11:47:22.038+0000 7f22bc8118c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:26.652 INFO:tasks.workunit.client.0.trial128.stderr:205400: 2026-02-17T11:47:22.038+0000 7f22bc8118c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:26.652 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T11:47:26.652 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:47:26.652 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T11:47:26.652 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:47:26.652 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:47:26.652 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:26.652 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:47:26.652 INFO:tasks.workunit.client.0.trial128.stderr:205400: 0 2026-02-17T11:47:26.652 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:26.652 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:47:26.652 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:26.653 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:26.653 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:26.653 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:47:26.653 INFO:tasks.workunit.client.0.trial128.stderr:205400: 1 2026-02-17T11:47:26.653 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:26.653 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:47:26.653 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:26.653 INFO:tasks.workunit.client.0.trial128.stderr:205400: 2026-02-17T11:47:23.578+0000 7f22bc8118c0 -1 Falling back to public interface 2026-02-17T11:47:26.653 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:26.653 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:26.653 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:47:26.654 INFO:tasks.workunit.client.0.trial128.stderr:205400: 2 2026-02-17T11:47:26.654 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:26.654 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:47:26.654 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:26.654 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:26.654 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:26.654 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:47:26.654 INFO:tasks.workunit.client.0.trial128.stderr:205400: 3 2026-02-17T11:47:26.654 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:26.654 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:47:26.654 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:27.297 INFO:tasks.workunit.client.0.trial128.stderr:205400: 2026-02-17T11:47:26.638+0000 7f2.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:27.298 INFO:tasks.workunit.client.0.trial128.stderr:205398: 2026-02-17T11:47:26.354+0000 7f0ba63708c0 -1 osd.3 106 log_to_monitors true 2026-02-17T11:47:27.298 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:27.298 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:27.298 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:47:27.298 INFO:tasks.workunit.client.0.trial128.stderr:205398: 4 2026-02-17T11:47:27.298 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:27.298 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:47:27.298 INFO:tasks.workunit.client.0.trial128.stderr:205398: osd.3 up in weight 1 up_from 111 up_thru 114 down_at 107 last_clean_interval [105,106) [v2:127.0.0.1:6826/751209474,v1:127.0.0.1:6827/751209474] [v2:127.0.0.1:6828/751209474,v1:127.0.0.1:6829/751209474] exists,up 3866ec67-d79b-45ae-83d0-612ac11a1b5b 2026-02-17T11:47:27.299 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:47:27.299 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:47:27.299 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:47:27.299 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:47:27.299 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:47:27.299 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:47:27.299 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:47:27.299 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:47:27.299 INFO:tasks.workunit.client.0.trial128.stderr:205398: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:47:27.299 INFO:tasks.workunit.client.0.trial128.stderr:205398: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:47:27.299 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:47:27.299 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:47:27.300 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:47:27.300 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:47:27.300 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:47:27.300 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:47:27.300 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:47:27.300 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:47:27.300 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:47:27.300 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:47:27.300 INFO:tasks.workunit.client.0.trial128.stderr:205398: 1 2026-02-17T11:47:27.300 INFO:tasks.workunit.client.0.trial128.stderr:205398: 2 2026-02-17T11:47:27.300 INFO:tasks.workunit.client.0.trial128.stderr:205398: 3 2026-02-17T11:47:27.301 INFO:tasks.workunit.client.0.trial128.stderr:205398: 4 2026-02-17T11:47:27.301 INFO:tasks.workunit.client.0.trial128.stderr:205398: 5 2026-02-17T11:47:27.301 INFO:tasks.workunit.client.0.trial128.stderr:205398: 6 2026-02-17T11:47:27.301 INFO:tasks.workunit.client.0.trial128.stderr:205398: 7 2026-02-17T11:47:27.301 INFO:tasks.workunit.client.0.trial128.stderr:205398: 8 2026-02-17T11:47:27.301 INFO:tasks.workunit.client.0.trial128.stderr:205398: 9' 2026-02-17T11:47:27.301 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:47:27.301 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:27.301 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:47:27.301 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836534 2026-02-17T11:47:27.301 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836534 2026-02-17T11:47:27.301 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534' 2026-02-17T11:47:27.302 INFO:tasks.workunit.client.0.trial128.stderr:205398: /2bc8118c0 -1 osd.5 106 log_to_monitors true 2026-02-17T11:47:27.302 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:27.302 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:27.302 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:47:27.302 INFO:tasks.workunit.client.0.trial128.stderr:205400: 4 2026-02-17T11:47:27.302 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:27.303 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:47:27.303 INFO:tasks.workunit.client.0.trial128.stderr:205400: osd.5 up in weight 1 up_from 114 up_thru 114 down_at 107 last_clean_interval [103,106) [v2:127.0.0.1:6842/250401649,v1:127.0.0.1:6843/250401649] [v2:127.0.0.1:6844/250401649,v1:127.0.0.1:6845/250401649] exists,up 64f61c82-c6d6-4640-bd33-5fd8a48c3bfa 2026-02-17T11:47:27.303 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:47:27.303 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:47:27.303 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:47:27.303 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:47:27.303 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:47:27.303 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:47:27.303 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:47:27.303 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:47:27.303 INFO:tasks.workunit.client.0.trial128.stderr:205400: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:47:27.304 INFO:tasks.workunit.client.0.trial128.stderr:205400: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:47:27.304 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:47:27.304 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:47:27.304 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:47:27.304 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:47:27.304 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:47:27.304 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:47:27.304 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:47:27.304 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:47:27.304 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:47:27.304 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:47:27.304 INFO:tasks.workunit.client.0.trial128.stderr:205400: 1 2026-02-17T11:47:27.305 INFO:tasks.workunit.client.0.trial128.stderr:205400: 2 2026-02-17T11:47:27.305 INFO:tasks.workunit.client.0.trial128.stderr:205400: 3 2026-02-17T11:47:27.305 INFO:tasks.workunit.client.0.trial128.stderr:205400: 4 2026-02-17T11:47:27.305 INFO:tasks.workunit.client.0.trial128.stderr:205400: 5 2026-02-17T11:47:27.305 INFO:tasks.workunit.client.0.trial128.stderr:205400: 6 2026-02-17T11:47:27.305 INFO:tasks.workunit.client.0.trial128.stderr:205400: 7 2026-02-17T11:47:27.305 INFO:tasks.workunit.client.0.trial128.stderr:205400: 8 2026-02-17T11:47:27.305 INFO:tasks.workunit.client.0.trial128.stderr:205400: 9' 2026-02-17T11:47:27.305 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:47:27.305 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:27.305 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:47:27.305 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836535 2026-02-17T11:47:27.306 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836535 2026-02-17T11:47:27.306 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535' 2026-02-17T11:47:27.733 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:27.733 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:47:27.733 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705718 2026-02-17T11:47:27.734 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705718 2026-02-17T11:47:27.734 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705718' 2026-02-17T11:47:27.734 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:27.734 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:47:27.734 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542195 2026-02-17T11:47:27.734 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542195 2026-02-17T11:47:27.734 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705718 2-60129542195' 2026-02-17T11:47:27.734 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:27.734 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:47:27.734 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=476741369859 2026-02-17T11:47:27.734 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 476741369859 2026-02-17T11:47:27.735 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705718 2-60129542195 3-476741369859' 2026-02-17T11:47:27.735 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:27.735 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:47:27.735 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247856 2026-02-17T11:47:27.735 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247856 2026-02-17T11:47:27.735 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705718 2-60129542195 3-476741369859 4-98784247856' 2026-02-17T11:47:27.735 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:27.735 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:47:27.735 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=489626271747 2026-02-17T11:47:27.735 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 489626271747 2026-02-17T11:47:27.735 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705718 2-60129542195 3-476741369859 4-98784247856 5-489626271747' 2026-02-17T11:47:27.735 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:27.736 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:47:27.736 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986221 2026-02-17T11:47:27.736 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986221 2026-02-17T11:47:27.736 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705718 2-60129542195 3-476741369859 4-98784247856 5-489626271747 6-133143986221' 2026-02-17T11:47:27.736 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:27.798 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tellids 2026-02-17T11:47:27.798 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:47:27.798 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705717 2026-02-17T11:47:27.798 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705717 2026-02-17T11:47:27.798 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705717' 2026-02-17T11:47:27.798 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:27.799 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:47:27.799 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542196 2026-02-17T11:47:27.799 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542196 2026-02-17T11:47:27.799 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705717 2-60129542196' 2026-02-17T11:47:27.799 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:27.799 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:47:27.799 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=476741369858 2026-02-17T11:47:27.799 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 476741369858 2026-02-17T11:47:27.799 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705717 2-60129542196 3-476741369858' 2026-02-17T11:47:27.799 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:27.799 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:47:27.799 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247855 2026-02-17T11:47:27.800 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247855 2026-02-17T11:47:27.800 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705717 2-60129542196 3-476741369858 4-98784247855' 2026-02-17T11:47:27.800 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:27.800 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:47:27.800 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=489626271746 2026-02-17T11:47:27.800 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 489626271746 2026-02-17T11:47:27.800 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705717 2-60129542196 3-476741369858 4-98784247855 5-489626271746' 2026-02-17T11:47:27.800 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:27.800 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:47:27.800 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986220 2026-02-17T11:47:27.800 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986220 2026-02-17T11:47:27.801 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705717 2-60129542196 3-476741369858 4-98784247855 5-489626271746 6-133143986220' 2026-02-17T11:47:27.801 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:27.801 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:47:29.258 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855402 2026-02-17T11:47:29.258 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855402 2026-02-17T11:47:29.258 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705717 2-60129542196 3-476741369858 4-98784247855 5-489626271746 6-133143986220 7-150323855402' 2026-02-17T11:47:29.259 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:29.259 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:47:29.259 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724585 2026-02-17T11:47:29.259 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724585 2026-02-17T11:47:29.259 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705717 2-60129542196 3-476741369858 4-98784247855 5-489626271746 6-133143986220 7-150323855402 8-167503724585' 2026-02-17T11:47:29.259 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:29.259 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:47:29.259 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=416611827724 2026-02-17T11:47:29.259 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 416611827724 2026-02-17T11:47:29.259 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705717 2-60129542196 3-476741369858 4-98784247855 5-489626271746 6-133143986220 7-150323855402 8-167503724585 9-416611827724' 2026-02-17T11:47:29.259 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:29.260 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836535 2026-02-17T11:47:29.260 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:29.260 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:47:29.260 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836535 2026-02-17T11:47:29.260 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:29.260 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836535 2026-02-17T11:47:29.260 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836535' 2026-02-17T11:47:29.260 INFO:tasks.workunit.client.0.trial128.stderr:205400: waiting osd.0 seq 21474836535 2026-02-17T11:47:29.260 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:47:29.260 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836533 -lt 21474836535 2026-02-17T11:47:29.260 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:47:29.260 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:47:29.261 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:47:29.261 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836536 -lt 21474836535 2026-02-17T11:47:29.261 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:29.261 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705717 2026-02-17T11:47:29.261 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:29.261 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:47:29.261 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705717 2026-02-17T11:47:29.263 INFO:tasks.workunit.client.0.trial128.stderr:205400: // osd.7 flush_pg_stats 2026-02-17T11:47:29.263 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855403 2026-02-17T11:47:29.263 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855403 2026-02-17T11:47:29.263 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705718 2-60129542195 3-476741369859 4-98784247856 5-489626271747 6-133143986221 7-150323855403' 2026-02-17T11:47:29.263 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:29.263 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:47:29.263 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724586 2026-02-17T11:47:29.263 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724586 2026-02-17T11:47:29.263 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705718 2-60129542195 3-476741369859 4-98784247856 5-489626271747 6-133143986221 7-150323855403 8-167503724586' 2026-02-17T11:47:29.264 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:47:29.264 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:47:29.264 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=416611827725 2026-02-17T11:47:29.264 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 416611827725 2026-02-17T11:47:29.264 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705718 2-60129542195 3-476741369859 4-98784247856 5-489626271747 6-133143986221 7-150323855403 8-167503724586 9-416611827725' 2026-02-17T11:47:29.264 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:29.264 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836534 2026-02-17T11:47:29.264 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:29.264 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:47:29.264 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836534 2026-02-17T11:47:29.264 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:29.264 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836534 2026-02-17T11:47:29.265 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836534' 2026-02-17T11:47:29.265 INFO:tasks.workunit.client.0.trial128.stderr:205398: waiting osd.0 seq 21474836534 2026-02-17T11:47:29.265 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:47:29.265 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836533 -lt 21474836534 2026-02-17T11:47:29.265 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:47:29.265 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:47:29.265 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:47:29.265 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836536 -lt 21474836534 2026-02-17T11:47:29.265 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:29.265 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705718 2026-02-17T11:47:29.265 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:29.265 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:47:29.741 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705718 2026-02-17T11:47:29.741 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:29.741 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705718 2026-02-17T11:47:29.741 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705718' 2026-02-17T11:47:29.741 INFO:tasks.workunit.client.0.trial128.stderr:205398: waiting osd.1 seq 38654705718 2026-02-17T11:47:29.741 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:47:29.741 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705718 -lt 38654705718 2026-02-17T11:47:29.741 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:29.742 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542195 2026-02-17T11:47:29.742 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:29.742 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:47:29.742 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542195 2026-02-17T11:47:29.742 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:29.742 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542195 2026-02-17T11:47:29.742 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542195' 2026-02-17T11:47:29.742 INFO:tasks.workunit.client.0.trial128.stderr:205398: waiting osd.2 seq 60129542195 2026-02-17T11:47:29.742 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:47:29.742 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542196 -lt 60129542195 2026-02-17T11:47:29.742 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:29.742 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-476741369859 2026-02-17T11:47:29.743 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:29.743 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:47:29.743 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-476741369859 2026-02-17T11:47:29.743 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:29.743 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=476741369859 2026-02-17T11:47:29.743 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 476741369859' 2026-02-17T11:47:29.743 INFO:tasks.workunit.client.0.trial128.stderr:205398: waiting osd.3 seq 476741369859 2026-02-17T11:47:29.743 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:47:29.743 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 476741369859 -lt 476741369859 2026-02-17T11:47:29.743 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:29.743 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247856 2026-02-17T11:47:29.743 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:29.744 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:47:29.744 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247856 2026-02-17T11:47:29.744 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:29.744 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247856 2026-02-17T11:47:29.750 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waithome/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:29.751 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705717 2026-02-17T11:47:29.751 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705717' 2026-02-17T11:47:29.751 INFO:tasks.workunit.client.0.trial128.stderr:205400: waiting osd.1 seq 38654705717 2026-02-17T11:47:29.751 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:47:29.751 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705718 -lt 38654705717 2026-02-17T11:47:29.751 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:29.751 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542196 2026-02-17T11:47:29.751 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:29.751 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:47:29.751 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542196 2026-02-17T11:47:29.751 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:29.751 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542196 2026-02-17T11:47:29.752 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542196' 2026-02-17T11:47:29.752 INFO:tasks.workunit.client.0.trial128.stderr:205400: waiting osd.2 seq 60129542196 2026-02-17T11:47:29.752 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:47:29.752 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542196 -lt 60129542196 2026-02-17T11:47:29.752 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:29.752 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-476741369858 2026-02-17T11:47:29.752 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:29.752 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:47:29.752 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-476741369858 2026-02-17T11:47:29.752 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:29.752 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=476741369858 2026-02-17T11:47:29.752 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 476741369858' 2026-02-17T11:47:29.753 INFO:tasks.workunit.client.0.trial128.stderr:205400: waiting osd.3 seq 476741369858 2026-02-17T11:47:29.753 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:47:29.753 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 476741369859 -lt 476741369858 2026-02-17T11:47:29.753 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:29.753 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247855 2026-02-17T11:47:29.753 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:29.753 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:47:29.753 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247855 2026-02-17T11:47:29.753 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:29.753 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247855 2026-02-17T11:47:29.753 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247855' 2026-02-17T11:47:29.753 INFO:tasks.workunit.client.0.trial128.stderr:205400: waiting osd.4 seq 98784247855 2026-02-17T11:47:30.353 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/cloneing osd.4 seq 98784247856' 2026-02-17T11:47:30.354 INFO:tasks.workunit.client.0.trial128.stderr:205398: waiting osd.4 seq 98784247856 2026-02-17T11:47:30.354 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:47:30.354 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247857 -lt 98784247856 2026-02-17T11:47:30.354 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:30.354 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-489626271747 2026-02-17T11:47:30.354 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:30.354 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:47:30.354 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-489626271747 2026-02-17T11:47:30.355 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:30.355 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=489626271747 2026-02-17T11:47:30.355 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 489626271747' 2026-02-17T11:47:30.355 INFO:tasks.workunit.client.0.trial128.stderr:205398: waiting osd.5 seq 489626271747 2026-02-17T11:47:30.355 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:47:30.355 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 489626271747 -lt 489626271747 2026-02-17T11:47:30.355 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:30.355 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986221 2026-02-17T11:47:30.355 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:30.355 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:47:30.356 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986221 2026-02-17T11:47:30.356 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:30.356 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986221 2026-02-17T11:47:30.356 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986221' 2026-02-17T11:47:30.356 INFO:tasks.workunit.client.0.trial128.stderr:205398: waiting osd.6 seq 133143986221 2026-02-17T11:47:30.356 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:47:30.356 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986221 -lt 133143986221 2026-02-17T11:47:30.356 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:30.356 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-150323855403 2026-02-17T11:47:30.357 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:30.357 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:47:30.357 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-150323855403 2026-02-17T11:47:30.357 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:30.357 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855403 2026-02-17T11:47:30.357 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 150323855403' 2026-02-17T11:47:30.357 INFO:tasks.workunit.client.0.trial128.stderr:205398: waiting osd.7 seq 150323855403 2026-02-17T11:47:30.357 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:47:30.357 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855403 -lt 150323855403 2026-02-17T11:47:30.389 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_st.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:47:30.389 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247857 -lt 98784247855 2026-02-17T11:47:30.390 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:30.390 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-489626271746 2026-02-17T11:47:30.390 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:30.390 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:47:30.390 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-489626271746 2026-02-17T11:47:30.390 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:30.390 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=489626271746 2026-02-17T11:47:30.391 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 489626271746' 2026-02-17T11:47:30.391 INFO:tasks.workunit.client.0.trial128.stderr:205400: waiting osd.5 seq 489626271746 2026-02-17T11:47:30.391 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:47:30.391 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 489626271747 -lt 489626271746 2026-02-17T11:47:30.391 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:30.391 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986220 2026-02-17T11:47:30.391 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:30.392 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:47:30.392 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986220 2026-02-17T11:47:30.392 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:30.392 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986220 2026-02-17T11:47:30.392 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986220' 2026-02-17T11:47:30.392 INFO:tasks.workunit.client.0.trial128.stderr:205400: waiting osd.6 seq 133143986220 2026-02-17T11:47:30.392 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:47:30.392 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986221 -lt 133143986220 2026-02-17T11:47:30.393 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:30.393 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-150323855402 2026-02-17T11:47:30.393 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:30.393 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:47:30.393 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-150323855402 2026-02-17T11:47:30.393 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:30.393 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855402 2026-02-17T11:47:30.393 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 150323855402' 2026-02-17T11:47:30.394 INFO:tasks.workunit.client.0.trial128.stderr:205400: waiting osd.7 seq 150323855402 2026-02-17T11:47:30.394 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:47:30.394 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855403 -lt 150323855402 2026-02-17T11:47:30.394 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:30.997 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274ats: for s in $seqs 2026-02-17T11:47:30.997 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-167503724586 2026-02-17T11:47:30.997 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:30.998 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:47:30.998 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724586 2026-02-17T11:47:30.998 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:30.998 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724586 2026-02-17T11:47:30.998 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724586' 2026-02-17T11:47:30.998 INFO:tasks.workunit.client.0.trial128.stderr:205398: waiting osd.8 seq 167503724586 2026-02-17T11:47:30.998 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:47:30.998 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724586 -lt 167503724586 2026-02-17T11:47:30.998 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:30.998 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-416611827725 2026-02-17T11:47:30.998 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:30.998 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:47:30.999 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-416611827725 2026-02-17T11:47:30.999 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:30.999 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=416611827725 2026-02-17T11:47:30.999 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 416611827725' 2026-02-17T11:47:30.999 INFO:tasks.workunit.client.0.trial128.stderr:205398: waiting osd.9 seq 416611827725 2026-02-17T11:47:30.999 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:47:30.999 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 416611827725 -lt 416611827725 2026-02-17T11:47:30.999 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:47:30.999 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:47:30.999 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:47:30.999 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:47:30.999 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:47:31.000 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:47:31.000 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:47:31.000 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:47:31.000 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:47:31.000 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:47:31.000 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:47:31.000 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:47:31.000 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:47:31.055 INFO:tasks.workunit.client.0.trial128.stderr:205398: //hom: flush_pg_stats: echo 8-167503724585 2026-02-17T11:47:31.055 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:31.055 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:47:31.056 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-167503724585 2026-02-17T11:47:31.056 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:31.056 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724585 2026-02-17T11:47:31.056 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 167503724585' 2026-02-17T11:47:31.056 INFO:tasks.workunit.client.0.trial128.stderr:205400: waiting osd.8 seq 167503724585 2026-02-17T11:47:31.056 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:47:31.056 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724586 -lt 167503724585 2026-02-17T11:47:31.056 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:47:31.056 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-416611827724 2026-02-17T11:47:31.056 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:47:31.056 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:47:31.056 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-416611827724 2026-02-17T11:47:31.057 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:47:31.057 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=416611827724 2026-02-17T11:47:31.057 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 416611827724' 2026-02-17T11:47:31.057 INFO:tasks.workunit.client.0.trial128.stderr:205400: waiting osd.9 seq 416611827724 2026-02-17T11:47:31.057 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:47:31.057 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 416611827725 -lt 416611827724 2026-02-17T11:47:31.057 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:47:31.057 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:47:31.057 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:47:31.057 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:47:31.057 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:47:31.057 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:47:31.058 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:47:31.058 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:47:31.058 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:47:31.058 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:47:31.058 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:47:31.058 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:47:31.058 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:47:31.173 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format jsone/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:47:31.173 INFO:tasks.workunit.client.0.trial128.stderr:205398: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:47:31.173 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:47:31.173 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:47:31.173 INFO:tasks.workunit.client.0.trial128.stderr:205398: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:47:31.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:47:31.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:47:31.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 205396 2026-02-17T11:47:31.244 INFO:tasks.workunit.client.0.trial128.stderr: status 2026-02-17T11:47:31.244 INFO:tasks.workunit.client.0.trial128.stderr:205400: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:47:31.244 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:47:31.244 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:47:31.244 INFO:tasks.workunit.client.0.trial128.stderr:205400: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:47:31.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:47:31.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T11:47:31.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T11:47:31.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T11:47:31.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:212: corrupt_and_repair_two: return_code=0 2026-02-17T11:47:31.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:213: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T11:47:31.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:47:31.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:47:31.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T11:47:31.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:47:31.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:47:31.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:47:31.267 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:47:31.267 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:47:31.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:47:31.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:47:31.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:47:31.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:47:31.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:47:31.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:47:31.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:47:31.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:47:31.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:47:31.584 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:47:31.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:47:31.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:47:31.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:47:31.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:47:31.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:47:31.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:47:31.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:47:31.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:47:31.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:47:31.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:47:31.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:47:31.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:47:31.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:47:31.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T11:47:31.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T11:47:31.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T11:47:31.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T11:47:31.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:47:31.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:47:31.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:47:31.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:47:31.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:47:31.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:47:31.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:47:31.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:47:31.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:47:31.636 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:47:31.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:47:31.636 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:47:31.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:47:31.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:47:31.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:47:31.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:47:31.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:47:31.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:47:31.640 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:47:31.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:47:31.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:47:31.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:47:31.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:47:31.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:47:31.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:47:31.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:47:31.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:47:31.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:47:31.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T11:47:31.647 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:47:31.647 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:47:31.647 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:47:31.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T11:47:31.649 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:47:31.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T11:47:31.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:47:31.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T11:47:31.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_and_repair_lrc_overwrites td/osd-scrub-repair 2026-02-17T11:47:31.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:846: TEST_corrupt_and_repair_lrc_overwrites: '[' true = true ']' 2026-02-17T11:47:31.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:847: TEST_corrupt_and_repair_lrc_overwrites: corrupt_and_repair_lrc td/osd-scrub-repair true 2026-02-17T11:47:31.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:825: corrupt_and_repair_lrc: local dir=td/osd-scrub-repair 2026-02-17T11:47:31.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:826: corrupt_and_repair_lrc: local allow_overwrites=true 2026-02-17T11:47:31.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:827: corrupt_and_repair_lrc: local poolname=ecpool 2026-02-17T11:47:31.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:829: corrupt_and_repair_lrc: run_mon td/osd-scrub-repair a 2026-02-17T11:47:31.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T11:47:31.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T11:47:31.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T11:47:31.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T11:47:31.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T11:47:31.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T11:47:31.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T11:47:31.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:47:31.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:47:31.672 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:47:31.672 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:47:31.672 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:47:31.672 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:31.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T11:47:31.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T11:47:31.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T11:47:31.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:47:31.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:47:31.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T11:47:31.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T11:47:31.711 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:47:31.711 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:47:31.711 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:47:31.712 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:47:31.712 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:47:31.712 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:47:31.712 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:47:31.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:47:31.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T11:47:31.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T11:47:31.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:47:31.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:47:31.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T11:47:31.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T11:47:31.770 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:47:31.770 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:47:31.770 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:47:31.770 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:47:31.770 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:47:31.770 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:47:31.770 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:47:31.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:47:31.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T11:47:31.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:830: corrupt_and_repair_lrc: run_mgr td/osd-scrub-repair x 2026-02-17T11:47:31.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T11:47:31.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T11:47:31.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T11:47:31.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T11:47:31.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T11:47:31.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T11:47:31.924 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T11:47:31.924 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:47:31.924 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:47:31.924 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:47:31.924 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:47:31.925 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:47:31.925 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:31.925 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:47:31.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:47:31.946 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: seq 0 9 2026-02-17T11:47:31.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:47:31.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 0 2026-02-17T11:47:31.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:47:31.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:47:31.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T11:47:31.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:47:31.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:47:31.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:47:31.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:47:31.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:47:31.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:47:31.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:47:31.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:47:31.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:47:31.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:47:31.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:47:31.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:47:31.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:47:31.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:47:31.951 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:47:31.951 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:47:31.951 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:47:31.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:31.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:31.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:47:31.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:47:31.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:47:31.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:47:31.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:47:31.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:47:31.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:47:31.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:47:31.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:47:31.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:47:31.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:47:31.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:47:31.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:47:31.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ac2035e9-a955-4326-94d6-22c621c7df03 2026-02-17T11:47:31.953 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 ac2035e9-a955-4326-94d6-22c621c7df03 2026-02-17T11:47:31.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 ac2035e9-a955-4326-94d6-22c621c7df03' 2026-02-17T11:47:31.954 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:47:31.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBTVZRpaQjAOBAAWKrswLW504CNZDiC22gcWQ== 2026-02-17T11:47:31.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBTVZRpaQjAOBAAWKrswLW504CNZDiC22gcWQ=="}' 2026-02-17T11:47:31.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ac2035e9-a955-4326-94d6-22c621c7df03 -i td/osd-scrub-repair/0/new.json 2026-02-17T11:47:32.049 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:47:32.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T11:47:32.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBTVZRpaQjAOBAAWKrswLW504CNZDiC22gcWQ== --osd-uuid ac2035e9-a955-4326-94d6-22c621c7df03 2026-02-17T11:47:32.072 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:32.058+0000 7fb2bd3a78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:32.074 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:32.058+0000 7fb2bd3a78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:32.075 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:32.062+0000 7fb2bd3a78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:32.076 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:32.062+0000 7fb2bd3a78c0 -1 bdev(0x555e00909c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:47:32.076 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:32.062+0000 7fb2bd3a78c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T11:47:34.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T11:47:34.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:47:34.625 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:47:34.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T11:47:34.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:47:34.826 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:47:34.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T11:47:34.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:47:34.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:47:34.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:47:34.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:47:34.849 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:34.834+0000 7fd9d0ea48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:34.851 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:34.838+0000 7fd9d0ea48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:34.852 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:34.838+0000 7fd9d0ea48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:34.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T11:47:34.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:47:34.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:47:34.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:47:34.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:47:34.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:34.978 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:47:34.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:47:34.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:34.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:47:35.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:36.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:36.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:36.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:47:36.135 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:47:36.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:36.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:47:36.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:36.352 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:36.338+0000 7fd9d0ea48c0 -1 Falling back to public interface 2026-02-17T11:47:37.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:37.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:37.291 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:47:37.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:47:37.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:37.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:47:37.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:38.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:38.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:38.447 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:47:38.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:47:38.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:38.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:47:38.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:39.334 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:39.318+0000 7fd9d0ea48c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:47:39.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:39.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:39.602 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:47:39.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:47:39.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:39.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:47:39.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:40.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:40.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:40.758 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:47:40.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:47:40.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:40.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:47:40.911 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1527199702,v1:127.0.0.1:6803/1527199702] [v2:127.0.0.1:6804/1527199702,v1:127.0.0.1:6805/1527199702] exists,up ac2035e9-a955-4326-94d6-22c621c7df03 2026-02-17T11:47:40.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:47:40.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:47:40.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:47:40.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:47:40.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 1 2026-02-17T11:47:40.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:47:40.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:47:40.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T11:47:40.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:47:40.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:47:40.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:47:40.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:47:40.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:47:40.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:47:40.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:47:40.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:47:40.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:47:40.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:47:40.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:47:40.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:47:40.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:47:40.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:47:40.917 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:47:40.917 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:47:40.917 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:47:40.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:40.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:40.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:47:40.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:47:40.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:47:40.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:47:40.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:47:40.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:47:40.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:47:40.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:47:40.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:47:40.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:47:40.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:47:40.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:47:40.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:47:40.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=1e5f3af3-4b91-403e-9111-71f6ce644a90 2026-02-17T11:47:40.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 1e5f3af3-4b91-403e-9111-71f6ce644a90' 2026-02-17T11:47:40.919 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 1e5f3af3-4b91-403e-9111-71f6ce644a90 2026-02-17T11:47:40.920 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:47:40.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBcVZRpYAHINhAA1/4MF7js0gyL10wii8ioZw== 2026-02-17T11:47:40.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBcVZRpYAHINhAA1/4MF7js0gyL10wii8ioZw=="}' 2026-02-17T11:47:40.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 1e5f3af3-4b91-403e-9111-71f6ce644a90 -i td/osd-scrub-repair/1/new.json 2026-02-17T11:47:41.086 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:47:41.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T11:47:41.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBcVZRpYAHINhAA1/4MF7js0gyL10wii8ioZw== --osd-uuid 1e5f3af3-4b91-403e-9111-71f6ce644a90 2026-02-17T11:47:41.112 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:41.098+0000 7f3126a658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:41.114 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:41.098+0000 7f3126a658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:41.115 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:41.102+0000 7f3126a658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:41.116 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:41.102+0000 7f3126a658c0 -1 bdev(0x559a88c23c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:47:41.116 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:41.102+0000 7f3126a658c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T11:47:43.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T11:47:43.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:47:43.406 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:47:43.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T11:47:43.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:47:43.603 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:47:43.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T11:47:43.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:47:43.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:47:43.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:47:43.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:47:43.625 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:43.610+0000 7f17237518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:43.626 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:43.614+0000 7f17237518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:43.627 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:43.614+0000 7f17237518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:43.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T11:47:43.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:47:43.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:47:43.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:47:43.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:47:43.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:43.755 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:47:43.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:47:43.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:43.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:47:43.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:44.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:44.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:44.910 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:47:44.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:47:44.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:44.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:47:45.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:45.124 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:45.110+0000 7f17237518c0 -1 Falling back to public interface 2026-02-17T11:47:46.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:46.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:46.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:47:46.067 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:47:46.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:46.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:47:46.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:47.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:47.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:47.223 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:47:47.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:47:47.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:47.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:47:47.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:48.121 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:48.106+0000 7f17237518c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:47:48.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:48.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:48.377 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:47:48.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:47:48.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:48.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:47:48.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:49.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:49.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:49.530 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:47:49.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:47:49.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:49.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:47:49.683 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/78407613,v1:127.0.0.1:6811/78407613] [v2:127.0.0.1:6812/78407613,v1:127.0.0.1:6813/78407613] exists,up 1e5f3af3-4b91-403e-9111-71f6ce644a90 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 2 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:47:49.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:47:49.685 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:47:49.685 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:47:49.685 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:47:49.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:49.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:49.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:47:49.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:47:49.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:47:49.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:47:49.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:47:49.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:47:49.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:47:49.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:47:49.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:47:49.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:47:49.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:47:49.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:47:49.687 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:47:49.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=31f9d49e-94ca-414b-bbc1-f7c8e091684e 2026-02-17T11:47:49.688 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 31f9d49e-94ca-414b-bbc1-f7c8e091684e 2026-02-17T11:47:49.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 31f9d49e-94ca-414b-bbc1-f7c8e091684e' 2026-02-17T11:47:49.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:47:49.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBlVZRpqAUyKRAAyQwhDsTQbnzlfLzRPrZ2jg== 2026-02-17T11:47:49.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBlVZRpqAUyKRAAyQwhDsTQbnzlfLzRPrZ2jg=="}' 2026-02-17T11:47:49.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 31f9d49e-94ca-414b-bbc1-f7c8e091684e -i td/osd-scrub-repair/2/new.json 2026-02-17T11:47:49.860 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:47:49.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T11:47:49.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBlVZRpqAUyKRAAyQwhDsTQbnzlfLzRPrZ2jg== --osd-uuid 31f9d49e-94ca-414b-bbc1-f7c8e091684e 2026-02-17T11:47:49.887 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:49.874+0000 7f81610a68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:49.889 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:49.874+0000 7f81610a68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:49.890 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:49.874+0000 7f81610a68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:49.890 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:49.874+0000 7f81610a68c0 -1 bdev(0x5595364cbc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T11:47:49.891 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:49.874+0000 7f81610a68c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T11:47:52.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T11:47:52.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:47:52.189 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T11:47:52.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T11:47:52.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:47:52.389 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T11:47:52.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T11:47:52.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:47:52.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:47:52.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:47:52.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:47:52.411 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:52.398+0000 7f78ccd788c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:52.413 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:52.398+0000 7f78ccd788c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:52.414 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:52.398+0000 7f78ccd788c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:52.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T11:47:52.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:47:52.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:47:52.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:47:52.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:47:52.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:52.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:47:52.541 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:47:52.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:52.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:47:52.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:53.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:53.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:53.697 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:47:53.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:47:53.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:53.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:47:53.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:53.928 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:53.913+0000 7f78ccd788c0 -1 Falling back to public interface 2026-02-17T11:47:54.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:54.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:54.856 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:47:54.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:47:54.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:54.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:47:55.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:56.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:56.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:56.013 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:47:56.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:47:56.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:56.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:47:56.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:56.940 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:56.929+0000 7f78ccd788c0 -1 osd.2 0 log_to_monitors true 2026-02-17T11:47:57.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:57.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:57.168 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:47:57.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:47:57.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:47:57.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:57.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:47:58.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:47:58.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:47:58.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:47:58.325 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:47:58.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:47:58.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:47:58.478 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4022883266,v1:127.0.0.1:6819/4022883266] [v2:127.0.0.1:6820/4022883266,v1:127.0.0.1:6821/4022883266] exists,up 31f9d49e-94ca-414b-bbc1-f7c8e091684e 2026-02-17T11:47:58.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:47:58.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:47:58.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:47:58.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:47:58.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 3 2026-02-17T11:47:58.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:47:58.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:47:58.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T11:47:58.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:47:58.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:47:58.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:47:58.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:47:58.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:47:58.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:47:58.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:47:58.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:47:58.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:47:58.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:47:58.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:47:58.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:47:58.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:47:58.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:47:58.485 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:47:58.486 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:47:58.486 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:47:58.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:58.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:47:58.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:47:58.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:47:58.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:47:58.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:47:58.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:47:58.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:47:58.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:47:58.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:47:58.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:47:58.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:47:58.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:47:58.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:47:58.487 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:47:58.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=063dd7f2-c316-40f3-bf48-6933c998a894 2026-02-17T11:47:58.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 063dd7f2-c316-40f3-bf48-6933c998a894' 2026-02-17T11:47:58.488 INFO:tasks.workunit.client.0.trial128.stdout:add osd3 063dd7f2-c316-40f3-bf48-6933c998a894 2026-02-17T11:47:58.488 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:47:58.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBuVZRplff7HBAAHMjsoWqx5xr8lOguCdzWwQ== 2026-02-17T11:47:58.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBuVZRplff7HBAAHMjsoWqx5xr8lOguCdzWwQ=="}' 2026-02-17T11:47:58.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 063dd7f2-c316-40f3-bf48-6933c998a894 -i td/osd-scrub-repair/3/new.json 2026-02-17T11:47:58.655 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:47:58.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-17T11:47:58.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBuVZRplff7HBAAHMjsoWqx5xr8lOguCdzWwQ== --osd-uuid 063dd7f2-c316-40f3-bf48-6933c998a894 2026-02-17T11:47:58.681 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:58.669+0000 7f1f18bde8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:58.682 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:58.669+0000 7f1f18bde8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:58.683 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:58.673+0000 7f1f18bde8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:47:58.684 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:58.673+0000 7f1f18bde8c0 -1 bdev(0x5561edd47c00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-17T11:47:58.684 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:47:58.673+0000 7f1f18bde8c0 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-17T11:48:00.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-17T11:48:00.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:48:00.968 INFO:tasks.workunit.client.0.trial128.stdout:adding osd3 key to auth repository 2026-02-17T11:48:00.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T11:48:00.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:48:01.165 INFO:tasks.workunit.client.0.trial128.stdout:start osd.3 2026-02-17T11:48:01.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T11:48:01.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:48:01.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:48:01.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:48:01.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:48:01.189 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:01.177+0000 7fcf298168c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:01.191 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:01.181+0000 7fcf298168c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:01.192 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:01.181+0000 7fcf298168c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:01.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T11:48:01.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:48:01.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:48:01.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:48:01.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:48:01.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:01.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:48:01.315 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:48:01.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:01.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:48:01.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:02.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:02.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:02.473 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:48:02.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:48:02.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:02.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:48:02.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:02.704 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:02.693+0000 7fcf298168c0 -1 Falling back to public interface 2026-02-17T11:48:03.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:03.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:03.628 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:48:03.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:48:03.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:03.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:48:03.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:04.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:04.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:04.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:48:04.783 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:48:04.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:04.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:48:04.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:05.759 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:05.749+0000 7fcf298168c0 -1 osd.3 0 log_to_monitors true 2026-02-17T11:48:05.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:05.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:05.938 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:48:05.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:48:05.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:05.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:48:06.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:07.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:07.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:07.096 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:48:07.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:48:07.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:07.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:48:07.250 INFO:tasks.workunit.client.0.trial128.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1263352967,v1:127.0.0.1:6827/1263352967] [v2:127.0.0.1:6828/1263352967,v1:127.0.0.1:6829/1263352967] exists,up 063dd7f2-c316-40f3-bf48-6933c998a894 2026-02-17T11:48:07.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:48:07.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:48:07.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:48:07.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:48:07.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 4 2026-02-17T11:48:07.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:48:07.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:48:07.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=4 2026-02-17T11:48:07.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:48:07.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/4 2026-02-17T11:48:07.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:48:07.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:48:07.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:48:07.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:48:07.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/4' 2026-02-17T11:48:07.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/4/journal' 2026-02-17T11:48:07.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:48:07.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:48:07.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:48:07.261 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:48:07.261 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:48:07.261 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:48:07.262 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:48:07.262 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:48:07.262 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:48:07.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:48:07.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:48:07.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:48:07.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:48:07.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:48:07.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:48:07.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:48:07.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:48:07.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:48:07.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:48:07.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:48:07.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:48:07.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:48:07.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/4 2026-02-17T11:48:07.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:48:07.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=d5e61ab6-f062-44c1-9079-9a67b8499fbd 2026-02-17T11:48:07.265 INFO:tasks.workunit.client.0.trial128.stdout:add osd4 d5e61ab6-f062-44c1-9079-9a67b8499fbd 2026-02-17T11:48:07.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd4 d5e61ab6-f062-44c1-9079-9a67b8499fbd' 2026-02-17T11:48:07.266 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:48:07.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB3VZRpJiBmDxAAJBg/vKIAYq5+WzTRWdxkKQ== 2026-02-17T11:48:07.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB3VZRpJiBmDxAAJBg/vKIAYq5+WzTRWdxkKQ=="}' 2026-02-17T11:48:07.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new d5e61ab6-f062-44c1-9079-9a67b8499fbd -i td/osd-scrub-repair/4/new.json 2026-02-17T11:48:07.425 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:48:07.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/4/new.json 2026-02-17T11:48:07.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 4 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/4 --osd-journal=td/osd-scrub-repair/4/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB3VZRpJiBmDxAAJBg/vKIAYq5+WzTRWdxkKQ== --osd-uuid d5e61ab6-f062-44c1-9079-9a67b8499fbd 2026-02-17T11:48:07.460 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:07.449+0000 7fb3669258c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:07.461 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:07.449+0000 7fb3669258c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:07.462 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:07.449+0000 7fb3669258c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:07.463 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:07.453+0000 7fb3669258c0 -1 bdev(0x56248a35dc00 td/osd-scrub-repair/4/block) open stat got: (1) Operation not permitted 2026-02-17T11:48:07.463 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:07.453+0000 7fb3669258c0 -1 bluestore(td/osd-scrub-repair/4) _read_fsid unparsable uuid 2026-02-17T11:48:09.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/4/keyring 2026-02-17T11:48:09.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:48:09.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd4 key to auth repository 2026-02-17T11:48:09.753 INFO:tasks.workunit.client.0.trial128.stdout:adding osd4 key to auth repository 2026-02-17T11:48:09.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:48:09.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.4 2026-02-17T11:48:09.950 INFO:tasks.workunit.client.0.trial128.stdout:start osd.4 2026-02-17T11:48:09.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 4 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/4 --osd-journal=td/osd-scrub-repair/4/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:48:09.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:48:09.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:48:09.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:48:09.972 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:09.961+0000 7f31da5708c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:09.974 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:09.961+0000 7f31da5708c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:09.975 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:09.965+0000 7f31da5708c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:10.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 4 2026-02-17T11:48:10.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:48:10.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=4 2026-02-17T11:48:10.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:48:10.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:48:10.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:10.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:48:10.100 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:48:10.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:10.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T11:48:10.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:11.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:11.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:11.255 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:48:11.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:48:11.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:11.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T11:48:11.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:11.496 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:11.485+0000 7f31da5708c0 -1 Falling back to public interface 2026-02-17T11:48:12.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:12.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:12.407 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:48:12.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:48:12.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:12.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T11:48:12.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:13.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:13.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:13.563 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:48:13.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:48:13.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:13.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T11:48:13.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:14.523 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:14.513+0000 7f31da5708c0 -1 osd.4 0 log_to_monitors true 2026-02-17T11:48:14.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:14.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:14.716 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:48:14.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:48:14.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:14.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T11:48:14.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:15.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:15.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:15.873 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:48:15.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:48:15.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:15.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.4 up' 2026-02-17T11:48:16.025 INFO:tasks.workunit.client.0.trial128.stdout:osd.4 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/3784541510,v1:127.0.0.1:6835/3784541510] [v2:127.0.0.1:6836/3784541510,v1:127.0.0.1:6837/3784541510] exists,up d5e61ab6-f062-44c1-9079-9a67b8499fbd 2026-02-17T11:48:16.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:48:16.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:48:16.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:48:16.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:48:16.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 5 2026-02-17T11:48:16.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:48:16.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:48:16.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=5 2026-02-17T11:48:16.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:48:16.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:48:16.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:48:16.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:48:16.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:48:16.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:48:16.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T11:48:16.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T11:48:16.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:48:16.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:48:16.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:48:16.030 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:48:16.030 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:48:16.030 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:48:16.030 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:48:16.031 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:48:16.031 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:48:16.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:48:16.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:48:16.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:48:16.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:48:16.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:48:16.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:48:16.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:48:16.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:48:16.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:48:16.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:48:16.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:48:16.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:48:16.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:48:16.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T11:48:16.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:48:16.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=80987ebb-2096-4a73-b7c5-64048030732e 2026-02-17T11:48:16.033 INFO:tasks.workunit.client.0.trial128.stdout:add osd5 80987ebb-2096-4a73-b7c5-64048030732e 2026-02-17T11:48:16.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd5 80987ebb-2096-4a73-b7c5-64048030732e' 2026-02-17T11:48:16.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:48:16.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCAVZRppKzxARAAxmGHLZ4cF8lFTrI85uhJqA== 2026-02-17T11:48:16.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCAVZRppKzxARAAxmGHLZ4cF8lFTrI85uhJqA=="}' 2026-02-17T11:48:16.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 80987ebb-2096-4a73-b7c5-64048030732e -i td/osd-scrub-repair/5/new.json 2026-02-17T11:48:16.203 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:48:16.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/5/new.json 2026-02-17T11:48:16.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 5 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCAVZRppKzxARAAxmGHLZ4cF8lFTrI85uhJqA== --osd-uuid 80987ebb-2096-4a73-b7c5-64048030732e 2026-02-17T11:48:16.229 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:16.217+0000 7fd21f66a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:16.230 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:16.217+0000 7fd21f66a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:16.231 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:16.221+0000 7fd21f66a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:16.232 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:16.221+0000 7fd21f66a8c0 -1 bdev(0x5636907ddc00 td/osd-scrub-repair/5/block) open stat got: (1) Operation not permitted 2026-02-17T11:48:16.232 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:16.221+0000 7fd21f66a8c0 -1 bluestore(td/osd-scrub-repair/5) _read_fsid unparsable uuid 2026-02-17T11:48:18.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/5/keyring 2026-02-17T11:48:18.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:48:18.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd5 key to auth repository 2026-02-17T11:48:18.518 INFO:tasks.workunit.client.0.trial128.stdout:adding osd5 key to auth repository 2026-02-17T11:48:18.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:48:18.708 INFO:tasks.workunit.client.0.trial128.stdout:start osd.5 2026-02-17T11:48:18.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.5 2026-02-17T11:48:18.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 5 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:48:18.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:48:18.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:48:18.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:48:18.730 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:18.717+0000 7f0729a898c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:18.732 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:18.721+0000 7f0729a898c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:18.733 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:18.721+0000 7f0729a898c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:18.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 5 2026-02-17T11:48:18.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:48:18.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T11:48:18.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:48:18.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:48:18.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:18.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:48:18.859 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:48:18.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:18.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:48:19.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:20.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:20.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:20.014 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:48:20.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:48:20.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:20.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:48:20.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:20.244 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:20.233+0000 7f0729a898c0 -1 Falling back to public interface 2026-02-17T11:48:21.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:21.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:21.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:48:21.166 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:48:21.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:21.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:48:21.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:22.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:22.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:22.328 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:48:22.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:48:22.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:22.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:48:22.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:23.290 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:23.277+0000 7f0729a898c0 -1 osd.5 0 log_to_monitors true 2026-02-17T11:48:23.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:23.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:23.480 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:48:23.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:48:23.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:23.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:48:23.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:24.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:24.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:24.639 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:48:24.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:48:24.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:24.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:48:24.795 INFO:tasks.workunit.client.0.trial128.stdout:osd.5 up in weight 1 up_from 27 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/3469717864,v1:127.0.0.1:6843/3469717864] [v2:127.0.0.1:6844/3469717864,v1:127.0.0.1:6845/3469717864] exists,up 80987ebb-2096-4a73-b7c5-64048030732e 2026-02-17T11:48:24.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:48:24.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:48:24.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:48:24.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:48:24.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 6 2026-02-17T11:48:24.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:48:24.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:48:24.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=6 2026-02-17T11:48:24.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:48:24.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/6 2026-02-17T11:48:24.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:48:24.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:48:24.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:48:24.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:48:24.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/6' 2026-02-17T11:48:24.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/6/journal' 2026-02-17T11:48:24.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:48:24.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:48:24.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:48:24.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:48:24.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:48:24.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/6 2026-02-17T11:48:24.798 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:48:24.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=349180dc-7f3f-41d4-a431-871bf7d075c0 2026-02-17T11:48:24.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd6 349180dc-7f3f-41d4-a431-871bf7d075c0' 2026-02-17T11:48:24.800 INFO:tasks.workunit.client.0.trial128.stdout:add osd6 349180dc-7f3f-41d4-a431-871bf7d075c0 2026-02-17T11:48:24.800 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:48:24.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCIVZRp36zeLxAA6YP8CIMVC8NoUC37Vfm0MQ== 2026-02-17T11:48:24.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCIVZRp36zeLxAA6YP8CIMVC8NoUC37Vfm0MQ=="}' 2026-02-17T11:48:24.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 349180dc-7f3f-41d4-a431-871bf7d075c0 -i td/osd-scrub-repair/6/new.json 2026-02-17T11:48:24.971 INFO:tasks.workunit.client.0.trial128.stdout:6 2026-02-17T11:48:24.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/6/new.json 2026-02-17T11:48:24.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 6 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/6 --osd-journal=td/osd-scrub-repair/6/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCIVZRp36zeLxAA6YP8CIMVC8NoUC37Vfm0MQ== --osd-uuid 349180dc-7f3f-41d4-a431-871bf7d075c0 2026-02-17T11:48:24.998 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:24.989+0000 7f72b51648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:25.000 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:24.989+0000 7f72b51648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:25.001 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:24.989+0000 7f72b51648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:25.001 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:24.989+0000 7f72b51648c0 -1 bdev(0x5571535bbc00 td/osd-scrub-repair/6/block) open stat got: (1) Operation not permitted 2026-02-17T11:48:25.002 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:24.989+0000 7f72b51648c0 -1 bluestore(td/osd-scrub-repair/6) _read_fsid unparsable uuid 2026-02-17T11:48:27.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/6/keyring 2026-02-17T11:48:27.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:48:27.273 INFO:tasks.workunit.client.0.trial128.stdout:adding osd6 key to auth repository 2026-02-17T11:48:27.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd6 key to auth repository 2026-02-17T11:48:27.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:48:27.463 INFO:tasks.workunit.client.0.trial128.stdout:start osd.6 2026-02-17T11:48:27.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.6 2026-02-17T11:48:27.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 6 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/6 --osd-journal=td/osd-scrub-repair/6/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:48:27.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:48:27.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:48:27.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:48:27.479 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:27.469+0000 7fc110fb48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:27.481 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:27.469+0000 7fc110fb48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:27.482 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:27.469+0000 7fc110fb48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:27.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 6 2026-02-17T11:48:27.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:48:27.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=6 2026-02-17T11:48:27.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:48:27.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:48:27.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:27.617 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:48:27.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:48:27.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:27.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T11:48:27.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:28.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:28.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:28.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:48:28.774 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:48:28.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:28.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T11:48:28.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:28.952 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:28.941+0000 7fc110fb48c0 -1 Falling back to public interface 2026-02-17T11:48:29.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:29.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:29.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:48:29.928 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:48:29.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:29.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T11:48:30.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:31.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:31.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:31.086 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:48:31.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:48:31.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:31.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T11:48:31.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:31.948 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:31.937+0000 7fc110fb48c0 -1 osd.6 0 log_to_monitors true 2026-02-17T11:48:32.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:32.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:32.238 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:48:32.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:48:32.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:32.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T11:48:32.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:33.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:33.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:33.393 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:48:33.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:48:33.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:33.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.6 up' 2026-02-17T11:48:33.544 INFO:tasks.workunit.client.0.trial128.stdout:osd.6 up in weight 1 up_from 31 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/1068394577,v1:127.0.0.1:6851/1068394577] [v2:127.0.0.1:6852/1068394577,v1:127.0.0.1:6853/1068394577] exists,up 349180dc-7f3f-41d4-a431-871bf7d075c0 2026-02-17T11:48:33.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:48:33.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:48:33.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:48:33.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:48:33.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 7 2026-02-17T11:48:33.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:48:33.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:48:33.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=7 2026-02-17T11:48:33.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:48:33.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/7 2026-02-17T11:48:33.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:48:33.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:48:33.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:48:33.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:48:33.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/7' 2026-02-17T11:48:33.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/7/journal' 2026-02-17T11:48:33.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:48:33.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:48:33.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:48:33.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:48:33.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:48:33.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:48:33.546 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:48:33.546 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:48:33.546 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:48:33.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:48:33.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:48:33.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:48:33.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:48:33.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:48:33.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:48:33.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:48:33.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:48:33.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:48:33.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:48:33.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:48:33.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:48:33.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:48:33.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/7 2026-02-17T11:48:33.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:48:33.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e7d1ab62-a895-4723-8868-3f1f8c858b8c 2026-02-17T11:48:33.549 INFO:tasks.workunit.client.0.trial128.stdout:add osd7 e7d1ab62-a895-4723-8868-3f1f8c858b8c 2026-02-17T11:48:33.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd7 e7d1ab62-a895-4723-8868-3f1f8c858b8c' 2026-02-17T11:48:33.549 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:48:33.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCRVZRpNwPmIBAAmiiOBbuSlQjXGUOy4WKvHw== 2026-02-17T11:48:33.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCRVZRpNwPmIBAAmiiOBbuSlQjXGUOy4WKvHw=="}' 2026-02-17T11:48:33.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e7d1ab62-a895-4723-8868-3f1f8c858b8c -i td/osd-scrub-repair/7/new.json 2026-02-17T11:48:33.719 INFO:tasks.workunit.client.0.trial128.stdout:7 2026-02-17T11:48:33.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/7/new.json 2026-02-17T11:48:33.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 7 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/7 --osd-journal=td/osd-scrub-repair/7/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCRVZRpNwPmIBAAmiiOBbuSlQjXGUOy4WKvHw== --osd-uuid e7d1ab62-a895-4723-8868-3f1f8c858b8c 2026-02-17T11:48:33.744 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:33.733+0000 7f6a85cec8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:33.745 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:33.733+0000 7f6a85cec8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:33.746 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:33.737+0000 7f6a85cec8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:33.747 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:33.737+0000 7f6a85cec8c0 -1 bdev(0x55acdbc5dc00 td/osd-scrub-repair/7/block) open stat got: (1) Operation not permitted 2026-02-17T11:48:33.747 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:33.737+0000 7f6a85cec8c0 -1 bluestore(td/osd-scrub-repair/7) _read_fsid unparsable uuid 2026-02-17T11:48:36.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/7/keyring 2026-02-17T11:48:36.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:48:36.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd7 key to auth repository 2026-02-17T11:48:36.025 INFO:tasks.workunit.client.0.trial128.stdout:adding osd7 key to auth repository 2026-02-17T11:48:36.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:48:36.219 INFO:tasks.workunit.client.0.trial128.stdout:start osd.7 2026-02-17T11:48:36.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.7 2026-02-17T11:48:36.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 7 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/7 --osd-journal=td/osd-scrub-repair/7/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:48:36.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:48:36.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:48:36.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:48:36.242 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:36.233+0000 7f9a0fc878c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:36.244 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:36.233+0000 7f9a0fc878c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:36.245 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:36.233+0000 7f9a0fc878c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:36.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 7 2026-02-17T11:48:36.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:48:36.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=7 2026-02-17T11:48:36.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:48:36.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:48:36.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:36.371 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:48:36.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:48:36.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:36.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T11:48:36.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:37.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:37.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:37.525 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:48:37.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:48:37.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:37.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T11:48:37.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:37.760 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:37.749+0000 7f9a0fc878c0 -1 Falling back to public interface 2026-02-17T11:48:38.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:38.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:38.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:48:38.682 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:48:38.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:38.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T11:48:38.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:39.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:39.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:39.835 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:48:39.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:48:39.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:39.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T11:48:39.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:40.776 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:40.765+0000 7f9a0fc878c0 -1 osd.7 0 log_to_monitors true 2026-02-17T11:48:40.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:40.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:40.989 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:48:40.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:48:40.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:40.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T11:48:41.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:42.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:42.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:42.141 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:48:42.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:48:42.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:42.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.7 up' 2026-02-17T11:48:42.291 INFO:tasks.workunit.client.0.trial128.stdout:osd.7 up in weight 1 up_from 36 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/2939892623,v1:127.0.0.1:6859/2939892623] [v2:127.0.0.1:6860/2939892623,v1:127.0.0.1:6861/2939892623] exists,up e7d1ab62-a895-4723-8868-3f1f8c858b8c 2026-02-17T11:48:42.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:48:42.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:48:42.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:48:42.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:48:42.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 8 2026-02-17T11:48:42.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:48:42.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:48:42.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=8 2026-02-17T11:48:42.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:48:42.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/8 2026-02-17T11:48:42.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:48:42.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:48:42.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:48:42.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:48:42.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/8' 2026-02-17T11:48:42.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/8/journal' 2026-02-17T11:48:42.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:48:42.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:48:42.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:48:42.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:48:42.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:48:42.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:48:42.297 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:48:42.297 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:48:42.297 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:48:42.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:48:42.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:48:42.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:48:42.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:48:42.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:48:42.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:48:42.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:48:42.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:48:42.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:48:42.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:48:42.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:48:42.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:48:42.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:48:42.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/8 2026-02-17T11:48:42.299 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:48:42.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=1344cde7-65d0-4d7e-bfad-6a055f456947 2026-02-17T11:48:42.299 INFO:tasks.workunit.client.0.trial128.stdout:add osd8 1344cde7-65d0-4d7e-bfad-6a055f456947 2026-02-17T11:48:42.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd8 1344cde7-65d0-4d7e-bfad-6a055f456947' 2026-02-17T11:48:42.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:48:42.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCaVZRpRbrYERAA+ypUY2YoX2Mo3ao8bm16xA== 2026-02-17T11:48:42.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCaVZRpRbrYERAA+ypUY2YoX2Mo3ao8bm16xA=="}' 2026-02-17T11:48:42.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 1344cde7-65d0-4d7e-bfad-6a055f456947 -i td/osd-scrub-repair/8/new.json 2026-02-17T11:48:42.468 INFO:tasks.workunit.client.0.trial128.stdout:8 2026-02-17T11:48:42.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/8/new.json 2026-02-17T11:48:42.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 8 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/8 --osd-journal=td/osd-scrub-repair/8/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCaVZRpRbrYERAA+ypUY2YoX2Mo3ao8bm16xA== --osd-uuid 1344cde7-65d0-4d7e-bfad-6a055f456947 2026-02-17T11:48:42.494 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:42.481+0000 7f07e828e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:42.495 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:42.485+0000 7f07e828e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:42.496 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:42.485+0000 7f07e828e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:42.497 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:42.485+0000 7f07e828e8c0 -1 bdev(0x558108169c00 td/osd-scrub-repair/8/block) open stat got: (1) Operation not permitted 2026-02-17T11:48:42.497 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:42.485+0000 7f07e828e8c0 -1 bluestore(td/osd-scrub-repair/8) _read_fsid unparsable uuid 2026-02-17T11:48:44.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/8/keyring 2026-02-17T11:48:44.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:48:44.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd8 key to auth repository 2026-02-17T11:48:44.764 INFO:tasks.workunit.client.0.trial128.stdout:adding osd8 key to auth repository 2026-02-17T11:48:44.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:48:44.961 INFO:tasks.workunit.client.0.trial128.stdout:start osd.8 2026-02-17T11:48:44.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.8 2026-02-17T11:48:44.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 8 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/8 --osd-journal=td/osd-scrub-repair/8/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:48:44.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:48:44.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:48:44.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:48:44.984 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:44.973+0000 7f56e532c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:44.986 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:44.973+0000 7f56e532c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:44.987 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:44.977+0000 7f56e532c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:45.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 8 2026-02-17T11:48:45.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:48:45.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=8 2026-02-17T11:48:45.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:48:45.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:48:45.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:45.112 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:48:45.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:48:45.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:45.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T11:48:45.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:46.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:46.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:46.266 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:48:46.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:48:46.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:46.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T11:48:46.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:46.504 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:46.493+0000 7f56e532c8c0 -1 Falling back to public interface 2026-02-17T11:48:47.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:47.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:47.417 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:48:47.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:48:47.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:47.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T11:48:47.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:48.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:48.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:48.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:48:48.568 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:48:48.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:48.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T11:48:48.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:49.526 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:49.513+0000 7f56e532c8c0 -1 osd.8 0 log_to_monitors true 2026-02-17T11:48:49.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:49.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:49.723 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:48:49.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:48:49.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:49.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T11:48:49.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:50.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:50.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:50.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:48:50.877 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:48:50.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:50.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.8 up' 2026-02-17T11:48:51.030 INFO:tasks.workunit.client.0.trial128.stdout:osd.8 up in weight 1 up_from 41 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/3390531417,v1:127.0.0.1:6867/3390531417] [v2:127.0.0.1:6868/3390531417,v1:127.0.0.1:6869/3390531417] exists,up 1344cde7-65d0-4d7e-bfad-6a055f456947 2026-02-17T11:48:51.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:48:51.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:48:51.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:48:51.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:831: corrupt_and_repair_lrc: for id in $(seq 0 9) 2026-02-17T11:48:51.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:832: corrupt_and_repair_lrc: run_osd td/osd-scrub-repair 9 2026-02-17T11:48:51.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:48:51.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:48:51.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=9 2026-02-17T11:48:51.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:48:51.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:48:51.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:48:51.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:48:51.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:48:51.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:48:51.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T11:48:51.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T11:48:51.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:48:51.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:48:51.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:48:51.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:48:51.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:48:51.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:48:51.036 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:48:51.036 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:48:51.036 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:48:51.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:48:51.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:48:51.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:48:51.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:48:51.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:48:51.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:48:51.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:48:51.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:48:51.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:48:51.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:48:51.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:48:51.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:48:51.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:48:51.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T11:48:51.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:48:51.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=6ca155d1-18e1-4577-a926-edebebb3c55d 2026-02-17T11:48:51.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd9 6ca155d1-18e1-4577-a926-edebebb3c55d' 2026-02-17T11:48:51.038 INFO:tasks.workunit.client.0.trial128.stdout:add osd9 6ca155d1-18e1-4577-a926-edebebb3c55d 2026-02-17T11:48:51.039 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:48:51.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCjVZRplzdPAhAAJeKOyxUljjmV0D/Y9syIKw== 2026-02-17T11:48:51.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCjVZRplzdPAhAAJeKOyxUljjmV0D/Y9syIKw=="}' 2026-02-17T11:48:51.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 6ca155d1-18e1-4577-a926-edebebb3c55d -i td/osd-scrub-repair/9/new.json 2026-02-17T11:48:51.199 INFO:tasks.workunit.client.0.trial128.stdout:9 2026-02-17T11:48:51.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/9/new.json 2026-02-17T11:48:51.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 9 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCjVZRplzdPAhAAJeKOyxUljjmV0D/Y9syIKw== --osd-uuid 6ca155d1-18e1-4577-a926-edebebb3c55d 2026-02-17T11:48:51.226 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:51.217+0000 7f89e440e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:51.228 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:51.217+0000 7f89e440e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:51.229 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:51.217+0000 7f89e440e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:51.229 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:51.217+0000 7f89e440e8c0 -1 bdev(0x563fd895fc00 td/osd-scrub-repair/9/block) open stat got: (1) Operation not permitted 2026-02-17T11:48:51.229 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:51.217+0000 7f89e440e8c0 -1 bluestore(td/osd-scrub-repair/9) _read_fsid unparsable uuid 2026-02-17T11:48:53.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/9/keyring 2026-02-17T11:48:53.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:48:53.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd9 key to auth repository 2026-02-17T11:48:53.524 INFO:tasks.workunit.client.0.trial128.stdout:adding osd9 key to auth repository 2026-02-17T11:48:53.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/9/keyring auth add osd.9 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:48:53.717 INFO:tasks.workunit.client.0.trial128.stdout:start osd.9 2026-02-17T11:48:53.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.9 2026-02-17T11:48:53.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 9 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:48:53.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:48:53.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:48:53.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:48:53.741 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:53.729+0000 7f8f925128c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:53.743 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:53.733+0000 7f8f925128c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:53.744 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:53.733+0000 7f8f925128c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:48:53.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 9 2026-02-17T11:48:53.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:48:53.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T11:48:53.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:48:53.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:48:53.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:53.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:48:53.868 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:48:53.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:53.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:48:54.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:55.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:55.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:55.024 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:48:55.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:48:55.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:55.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:48:55.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:55.239 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:55.229+0000 7f8f925128c0 -1 Falling back to public interface 2026-02-17T11:48:56.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:56.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:56.174 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:48:56.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:48:56.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:56.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:48:56.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:57.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:57.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:57.328 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:48:57.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:48:57.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:57.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:48:57.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:58.248 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:48:58.237+0000 7f8f925128c0 -1 osd.9 0 log_to_monitors true 2026-02-17T11:48:58.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:58.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:58.480 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:48:58.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:48:58.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:58.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:48:58.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:48:59.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:48:59.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:48:59.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:48:59.635 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:48:59.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:48:59.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:48:59.791 INFO:tasks.workunit.client.0.trial128.stdout:osd.9 up in weight 1 up_from 45 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6874/3904314399,v1:127.0.0.1:6875/3904314399] [v2:127.0.0.1:6876/3904314399,v1:127.0.0.1:6877/3904314399] exists,up 6ca155d1-18e1-4577-a926-edebebb3c55d 2026-02-17T11:48:59.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:48:59.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:48:59.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:48:59.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:834: corrupt_and_repair_lrc: create_rbd_pool 2026-02-17T11:48:59.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T11:48:59.935 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' does not exist 2026-02-17T11:48:59.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T11:48:59.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T11:49:00.156 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T11:49:00.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:49:01.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T11:49:01.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:835: corrupt_and_repair_lrc: wait_for_clean 2026-02-17T11:49:01.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:49:01.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:49:01.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:49:01.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:49:01.450 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:49:01.450 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:49:01.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:49:01.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:49:01.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:49:01.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:49:01.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:49:01.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:49:01.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:49:01.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:49:01.562 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:49:01.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:49:01.720 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:49:01.720 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:49:01.720 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:49:01.720 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:49:01.720 INFO:tasks.workunit.client.0.trial128.stderr:5 2026-02-17T11:49:01.720 INFO:tasks.workunit.client.0.trial128.stderr:6 2026-02-17T11:49:01.720 INFO:tasks.workunit.client.0.trial128.stderr:7 2026-02-17T11:49:01.720 INFO:tasks.workunit.client.0.trial128.stderr:8 2026-02-17T11:49:01.720 INFO:tasks.workunit.client.0.trial128.stderr:9' 2026-02-17T11:49:01.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:49:01.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:01.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:49:01.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836498 2026-02-17T11:49:01.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836498 2026-02-17T11:49:01.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498' 2026-02-17T11:49:01.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:01.793 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:49:01.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705680 2026-02-17T11:49:01.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705680 2026-02-17T11:49:01.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680' 2026-02-17T11:49:01.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:01.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:49:01.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574862 2026-02-17T11:49:01.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574862 2026-02-17T11:49:01.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-55834574862' 2026-02-17T11:49:01.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:01.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:49:02.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411341 2026-02-17T11:49:02.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411341 2026-02-17T11:49:02.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-55834574862 3-77309411341' 2026-02-17T11:49:02.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:02.008 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:49:02.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247819 2026-02-17T11:49:02.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247819 2026-02-17T11:49:02.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-55834574862 3-77309411341 4-98784247819' 2026-02-17T11:49:02.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:02.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:49:02.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964117001 2026-02-17T11:49:02.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964117001 2026-02-17T11:49:02.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-55834574862 3-77309411341 4-98784247819 5-115964117001' 2026-02-17T11:49:02.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:02.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:49:02.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986183 2026-02-17T11:49:02.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986183 2026-02-17T11:49:02.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-55834574862 3-77309411341 4-98784247819 5-115964117001 6-133143986183' 2026-02-17T11:49:02.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:02.227 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:49:02.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822662 2026-02-17T11:49:02.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822662 2026-02-17T11:49:02.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-55834574862 3-77309411341 4-98784247819 5-115964117001 6-133143986183 7-154618822662' 2026-02-17T11:49:02.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:02.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:49:02.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659140 2026-02-17T11:49:02.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659140 2026-02-17T11:49:02.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-55834574862 3-77309411341 4-98784247819 5-115964117001 6-133143986183 7-154618822662 8-176093659140' 2026-02-17T11:49:02.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:02.372 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:49:02.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=193273528322 2026-02-17T11:49:02.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 193273528322 2026-02-17T11:49:02.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705680 2-55834574862 3-77309411341 4-98784247819 5-115964117001 6-133143986183 7-154618822662 8-176093659140 9-193273528322' 2026-02-17T11:49:02.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:02.443 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836498 2026-02-17T11:49:02.444 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:02.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:49:02.445 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836498 2026-02-17T11:49:02.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:02.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836498 2026-02-17T11:49:02.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836498' 2026-02-17T11:49:02.447 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836498 2026-02-17T11:49:02.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:49:02.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836498 -lt 21474836498 2026-02-17T11:49:02.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:02.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705680 2026-02-17T11:49:02.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:02.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:49:02.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705680 2026-02-17T11:49:02.604 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:02.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705680 2026-02-17T11:49:02.605 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705680 2026-02-17T11:49:02.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705680' 2026-02-17T11:49:02.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:49:02.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705680 -lt 38654705680 2026-02-17T11:49:02.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:02.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574862 2026-02-17T11:49:02.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:02.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:49:02.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574862 2026-02-17T11:49:02.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:02.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574862 2026-02-17T11:49:02.767 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574862 2026-02-17T11:49:02.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574862' 2026-02-17T11:49:02.767 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:49:02.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574862 -lt 55834574862 2026-02-17T11:49:02.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:02.925 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-77309411341 2026-02-17T11:49:02.925 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:02.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:49:02.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-77309411341 2026-02-17T11:49:02.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:02.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411341 2026-02-17T11:49:02.929 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.3 seq 77309411341 2026-02-17T11:49:02.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 77309411341' 2026-02-17T11:49:02.929 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:49:03.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411341 -lt 77309411341 2026-02-17T11:49:03.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:03.092 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247819 2026-02-17T11:49:03.092 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:03.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:49:03.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247819 2026-02-17T11:49:03.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:03.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247819 2026-02-17T11:49:03.096 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.4 seq 98784247819 2026-02-17T11:49:03.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247819' 2026-02-17T11:49:03.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:49:03.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247819 -lt 98784247819 2026-02-17T11:49:03.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:03.260 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-115964117001 2026-02-17T11:49:03.260 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:03.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:49:03.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-115964117001 2026-02-17T11:49:03.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:03.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964117001 2026-02-17T11:49:03.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 115964117001' 2026-02-17T11:49:03.264 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.5 seq 115964117001 2026-02-17T11:49:03.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:49:03.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964117001 -lt 115964117001 2026-02-17T11:49:03.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:03.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986183 2026-02-17T11:49:03.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:03.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:49:03.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986183 2026-02-17T11:49:03.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:03.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986183 2026-02-17T11:49:03.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986183' 2026-02-17T11:49:03.420 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.6 seq 133143986183 2026-02-17T11:49:03.421 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:49:03.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986182 -lt 133143986183 2026-02-17T11:49:03.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:49:04.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:49:04.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:49:04.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986184 -lt 133143986183 2026-02-17T11:49:04.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:04.742 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822662 2026-02-17T11:49:04.742 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:04.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:49:04.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822662 2026-02-17T11:49:04.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:04.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822662 2026-02-17T11:49:04.746 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.7 seq 154618822662 2026-02-17T11:49:04.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822662' 2026-02-17T11:49:04.746 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:49:04.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822662 -lt 154618822662 2026-02-17T11:49:04.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:04.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659140 2026-02-17T11:49:04.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:04.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:49:04.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659140 2026-02-17T11:49:04.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:04.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659140 2026-02-17T11:49:04.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659140' 2026-02-17T11:49:04.903 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.8 seq 176093659140 2026-02-17T11:49:04.903 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:49:05.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659140 -lt 176093659140 2026-02-17T11:49:05.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:05.057 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-193273528322 2026-02-17T11:49:05.057 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:05.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:49:05.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-193273528322 2026-02-17T11:49:05.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:05.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=193273528322 2026-02-17T11:49:05.061 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.9 seq 193273528322 2026-02-17T11:49:05.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 193273528322' 2026-02-17T11:49:05.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:49:05.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528322 -lt 193273528322 2026-02-17T11:49:05.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:49:05.227 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:49:05.227 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:49:05.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T11:49:05.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:49:05.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:49:05.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:49:05.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:49:05.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:49:05.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:49:05.415 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:49:05.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T11:49:05.566 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:49:05.567 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:49:05.567 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:49:05.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T11:49:05.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:49:05.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:49:05.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:837: corrupt_and_repair_lrc: create_ec_pool ecpool true k=4 m=2 l=3 plugin=lrc 2026-02-17T11:49:05.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T11:49:05.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T11:49:05.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=true 2026-02-17T11:49:05.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T11:49:05.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=4 m=2 l=3 plugin=lrc 2026-02-17T11:49:05.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T11:49:05.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T11:49:06.258 INFO:tasks.workunit.client.0.trial128.stderr:pool 'ecpool' created 2026-02-17T11:49:06.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:49:07.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' true = true ']' 2026-02-17T11:49:07.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2508: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-17T11:49:07.464 INFO:tasks.workunit.client.0.trial128.stderr:set pool 2 allow_ec_overwrites to true 2026-02-17T11:49:07.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T11:49:07.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:49:07.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:49:07.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:49:07.474 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:49:07.475 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:49:07.475 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:49:07.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:49:07.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:49:07.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:49:07.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:49:07.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:49:07.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:49:07.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:49:07.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:49:07.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:49:07.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:49:07.748 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:49:07.748 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:49:07.748 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:49:07.748 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:49:07.748 INFO:tasks.workunit.client.0.trial128.stderr:5 2026-02-17T11:49:07.748 INFO:tasks.workunit.client.0.trial128.stderr:6 2026-02-17T11:49:07.749 INFO:tasks.workunit.client.0.trial128.stderr:7 2026-02-17T11:49:07.749 INFO:tasks.workunit.client.0.trial128.stderr:8 2026-02-17T11:49:07.749 INFO:tasks.workunit.client.0.trial128.stderr:9' 2026-02-17T11:49:07.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:49:07.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:07.749 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:49:07.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836500 2026-02-17T11:49:07.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836500 2026-02-17T11:49:07.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500' 2026-02-17T11:49:07.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:07.820 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:49:07.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705682 2026-02-17T11:49:07.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705682 2026-02-17T11:49:07.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682' 2026-02-17T11:49:07.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:07.892 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:49:07.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574864 2026-02-17T11:49:07.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574864 2026-02-17T11:49:07.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-55834574864' 2026-02-17T11:49:07.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:07.964 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:49:08.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411343 2026-02-17T11:49:08.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411343 2026-02-17T11:49:08.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-55834574864 3-77309411343' 2026-02-17T11:49:08.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:08.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:49:08.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247821 2026-02-17T11:49:08.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247821 2026-02-17T11:49:08.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-55834574864 3-77309411343 4-98784247821' 2026-02-17T11:49:08.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:08.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:49:08.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964117003 2026-02-17T11:49:08.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964117003 2026-02-17T11:49:08.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-55834574864 3-77309411343 4-98784247821 5-115964117003' 2026-02-17T11:49:08.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:08.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:49:08.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986186 2026-02-17T11:49:08.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986186 2026-02-17T11:49:08.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-55834574864 3-77309411343 4-98784247821 5-115964117003 6-133143986186' 2026-02-17T11:49:08.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:08.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:49:08.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822664 2026-02-17T11:49:08.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822664 2026-02-17T11:49:08.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-55834574864 3-77309411343 4-98784247821 5-115964117003 6-133143986186 7-154618822664' 2026-02-17T11:49:08.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:08.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:49:08.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659142 2026-02-17T11:49:08.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659142 2026-02-17T11:49:08.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-55834574864 3-77309411343 4-98784247821 5-115964117003 6-133143986186 7-154618822664 8-176093659142' 2026-02-17T11:49:08.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:08.396 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:49:08.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=193273528324 2026-02-17T11:49:08.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 193273528324 2026-02-17T11:49:08.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500 1-38654705682 2-55834574864 3-77309411343 4-98784247821 5-115964117003 6-133143986186 7-154618822664 8-176093659142 9-193273528324' 2026-02-17T11:49:08.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:08.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836500 2026-02-17T11:49:08.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:08.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:49:08.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836500 2026-02-17T11:49:08.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:08.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836500 2026-02-17T11:49:08.473 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836500 2026-02-17T11:49:08.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836500' 2026-02-17T11:49:08.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:49:08.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836500 -lt 21474836500 2026-02-17T11:49:08.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:08.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705682 2026-02-17T11:49:08.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:08.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:49:08.635 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705682 2026-02-17T11:49:08.636 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:08.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705682 2026-02-17T11:49:08.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705682' 2026-02-17T11:49:08.637 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705682 2026-02-17T11:49:08.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:49:08.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705682 -lt 38654705682 2026-02-17T11:49:08.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:08.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574864 2026-02-17T11:49:08.800 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:08.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:49:08.801 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574864 2026-02-17T11:49:08.801 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:08.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574864 2026-02-17T11:49:08.803 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574864 2026-02-17T11:49:08.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574864' 2026-02-17T11:49:08.804 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:49:08.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574865 -lt 55834574864 2026-02-17T11:49:08.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:08.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-77309411343 2026-02-17T11:49:08.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:08.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:49:08.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-77309411343 2026-02-17T11:49:08.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:08.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411343 2026-02-17T11:49:08.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 77309411343' 2026-02-17T11:49:08.959 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.3 seq 77309411343 2026-02-17T11:49:08.959 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:49:09.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411343 -lt 77309411343 2026-02-17T11:49:09.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:09.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247821 2026-02-17T11:49:09.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:09.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:49:09.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247821 2026-02-17T11:49:09.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:09.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247821 2026-02-17T11:49:09.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247821' 2026-02-17T11:49:09.112 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.4 seq 98784247821 2026-02-17T11:49:09.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:49:09.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247821 -lt 98784247821 2026-02-17T11:49:09.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:09.272 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-115964117003 2026-02-17T11:49:09.272 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:09.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:49:09.274 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-115964117003 2026-02-17T11:49:09.274 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:09.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964117003 2026-02-17T11:49:09.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 115964117003' 2026-02-17T11:49:09.276 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.5 seq 115964117003 2026-02-17T11:49:09.276 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:49:09.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964117003 -lt 115964117003 2026-02-17T11:49:09.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:09.437 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986186 2026-02-17T11:49:09.437 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:09.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:49:09.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986186 2026-02-17T11:49:09.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:09.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986186 2026-02-17T11:49:09.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986186' 2026-02-17T11:49:09.440 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.6 seq 133143986186 2026-02-17T11:49:09.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:49:09.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986185 -lt 133143986186 2026-02-17T11:49:09.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:49:10.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:49:10.605 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:49:10.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986186 -lt 133143986186 2026-02-17T11:49:10.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:10.756 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822664 2026-02-17T11:49:10.756 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:10.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:49:10.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822664 2026-02-17T11:49:10.759 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:10.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822664 2026-02-17T11:49:10.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822664' 2026-02-17T11:49:10.760 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.7 seq 154618822664 2026-02-17T11:49:10.761 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:49:10.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822664 -lt 154618822664 2026-02-17T11:49:10.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:10.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659142 2026-02-17T11:49:10.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:10.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:49:10.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659142 2026-02-17T11:49:10.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:10.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659142 2026-02-17T11:49:10.919 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.8 seq 176093659142 2026-02-17T11:49:10.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659142' 2026-02-17T11:49:10.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:49:11.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659142 -lt 176093659142 2026-02-17T11:49:11.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:11.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-193273528324 2026-02-17T11:49:11.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:11.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:49:11.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-193273528324 2026-02-17T11:49:11.074 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:11.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=193273528324 2026-02-17T11:49:11.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 193273528324' 2026-02-17T11:49:11.075 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.9 seq 193273528324 2026-02-17T11:49:11.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:49:11.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528325 -lt 193273528324 2026-02-17T11:49:11.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:49:11.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:49:11.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:49:11.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:49:11.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:49:11.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:49:11.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:49:11.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:49:11.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:49:11.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:49:11.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:49:11.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:49:11.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:49:11.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:49:11.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:49:11.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:49:11.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:49:11.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:49:11.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T11:49:11.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:838: corrupt_and_repair_lrc: corrupt_and_repair_erasure_coded td/osd-scrub-repair ecpool 2026-02-17T11:49:11.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:248: corrupt_and_repair_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T11:49:11.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:249: corrupt_and_repair_erasure_coded: local poolname=ecpool 2026-02-17T11:49:11.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:251: corrupt_and_repair_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-17T11:49:11.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:49:11.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:49:11.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T11:49:11.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:49:11.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:49:11.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:49:11.982 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:49:11.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:49:12.187 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:49:12.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:49:12.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:49:12.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T11:49:12.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_erasure_coded: get_primary ecpool SOMETHING 2026-02-17T11:49:12.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T11:49:12.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:49:12.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:49:12.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:49:12.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:253: corrupt_and_repair_erasure_coded: local primary=3 2026-02-17T11:49:12.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: get_osds ecpool SOMETHING 2026-02-17T11:49:12.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T11:49:12.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: sed -e s/3// 2026-02-17T11:49:12.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T11:49:12.382 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:49:12.382 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T11:49:12.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=3 2026-02-17T11:49:12.539 INFO:tasks.workunit.client.0.trial128.stderr:5 2026-02-17T11:49:12.539 INFO:tasks.workunit.client.0.trial128.stderr:9 2026-02-17T11:49:12.539 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:49:12.540 INFO:tasks.workunit.client.0.trial128.stderr:6 2026-02-17T11:49:12.540 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:49:12.540 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:49:12.540 INFO:tasks.workunit.client.0.trial128.stderr:7' 2026-02-17T11:49:12.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 3 5 9 0 6 2 1 7 2026-02-17T11:49:12.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: osds=('5' '9' '0' '6' '2' '1' '7') 2026-02-17T11:49:12.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:254: corrupt_and_repair_erasure_coded: local -a osds 2026-02-17T11:49:12.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:255: corrupt_and_repair_erasure_coded: local not_primary_first=5 2026-02-17T11:49:12.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:256: corrupt_and_repair_erasure_coded: local not_primary_second=9 2026-02-17T11:49:12.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:259: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 3 2026-02-17T11:49:12.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T11:49:12.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=ecpool 2026-02-17T11:49:12.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=3 2026-02-17T11:49:12.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:49:12.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:49:12.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:49:12.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T11:49:12.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:49:12.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:49:12.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:49:12.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:49:12.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T11:49:12.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:49:12.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T11:49:12.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:49:12.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:49:12.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:49:12.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:49:12.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:49:13.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:49:13.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:49:13.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:49:13.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:49:13.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T11:49:13.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:49:13.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:49:13.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T11:49:14.502 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#2:eb822e21:::SOMETHING:head# 2026-02-17T11:49:15.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T11:49:15.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:49:15.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:49:15.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T11:49:15.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:49:15.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:49:15.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:49:15.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:49:15.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:49:15.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:49:15.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:49:15.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:49:15.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:49:15.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:49:15.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:49:15.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:49:15.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:49:15.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:49:15.039 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:49:15.039 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:49:15.039 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:49:15.039 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:49:15.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:49:15.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:49:15.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:49:15.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:49:15.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:49:15.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:49:15.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:49:15.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:49:15.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:49:15.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:49:15.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:49:15.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T11:49:15.041 INFO:tasks.workunit.client.0.trial128.stderr:start osd.3 2026-02-17T11:49:15.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:49:15.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T11:49:15.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T11:49:15.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:49:15.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:49:15.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:49:15.056 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:15.045+0000 7f43941518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:49:15.058 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:15.045+0000 7f43941518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:49:15.058 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:15.049+0000 7f43941518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:49:15.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T11:49:15.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:49:15.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:49:15.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:49:15.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:49:15.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:49:15.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:49:15.189 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:49:15.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:49:15.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:49:15.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:49:16.300 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:16.289+0000 7f43941518c0 -1 Falling back to public interface 2026-02-17T11:49:16.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:49:16.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:49:16.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:49:16.347 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:49:16.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:49:16.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:49:16.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:49:17.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:49:17.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:49:17.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:49:17.502 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:49:17.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:49:17.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:49:17.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:49:18.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:49:18.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:49:18.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:49:18.659 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:49:18.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:49:18.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:49:18.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:49:19.333 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:19.321+0000 7f43941518c0 -1 osd.3 60 log_to_monitors true 2026-02-17T11:49:19.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:49:19.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:49:19.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:49:19.818 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:49:19.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:49:19.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:49:19.974 INFO:tasks.workunit.client.0.trial128.stderr:osd.3 up in weight 1 up_from 64 up_thru 64 down_at 61 last_clean_interval [18,60) [v2:127.0.0.1:6826/2236696359,v1:127.0.0.1:6827/2236696359] [v2:127.0.0.1:6828/2236696359,v1:127.0.0.1:6829/2236696359] exists,up 063dd7f2-c316-40f3-bf48-6933c998a894 2026-02-17T11:49:19.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:49:19.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:49:19.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:49:19.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:49:19.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:49:19.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:49:19.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:49:19.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:49:19.976 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:49:19.976 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:49:19.976 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:49:19.976 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:49:19.976 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:49:20.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:49:20.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:49:20.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:49:20.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:49:20.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:49:20.089 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:49:20.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:49:20.246 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:49:20.246 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:49:20.246 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:49:20.246 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:49:20.246 INFO:tasks.workunit.client.0.trial128.stderr:5 2026-02-17T11:49:20.246 INFO:tasks.workunit.client.0.trial128.stderr:6 2026-02-17T11:49:20.246 INFO:tasks.workunit.client.0.trial128.stderr:7 2026-02-17T11:49:20.246 INFO:tasks.workunit.client.0.trial128.stderr:8 2026-02-17T11:49:20.246 INFO:tasks.workunit.client.0.trial128.stderr:9' 2026-02-17T11:49:20.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:49:20.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:20.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:49:20.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836503 2026-02-17T11:49:20.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836503 2026-02-17T11:49:20.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503' 2026-02-17T11:49:20.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:20.319 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:49:20.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705686 2026-02-17T11:49:20.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705686 2026-02-17T11:49:20.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686' 2026-02-17T11:49:20.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:20.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:49:20.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574868 2026-02-17T11:49:20.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574868 2026-02-17T11:49:20.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574868' 2026-02-17T11:49:20.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:20.464 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:49:20.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=274877906946 2026-02-17T11:49:20.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 274877906946 2026-02-17T11:49:20.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574868 3-274877906946' 2026-02-17T11:49:20.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:20.537 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:49:20.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247824 2026-02-17T11:49:20.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247824 2026-02-17T11:49:20.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574868 3-274877906946 4-98784247824' 2026-02-17T11:49:20.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:20.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:49:20.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964117007 2026-02-17T11:49:20.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964117007 2026-02-17T11:49:20.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574868 3-274877906946 4-98784247824 5-115964117007' 2026-02-17T11:49:20.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:20.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:49:20.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986189 2026-02-17T11:49:20.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986189 2026-02-17T11:49:20.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574868 3-274877906946 4-98784247824 5-115964117007 6-133143986189' 2026-02-17T11:49:20.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:20.752 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:49:20.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822667 2026-02-17T11:49:20.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822667 2026-02-17T11:49:20.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574868 3-274877906946 4-98784247824 5-115964117007 6-133143986189 7-154618822667' 2026-02-17T11:49:20.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:20.821 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:49:20.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659146 2026-02-17T11:49:20.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659146 2026-02-17T11:49:20.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574868 3-274877906946 4-98784247824 5-115964117007 6-133143986189 7-154618822667 8-176093659146' 2026-02-17T11:49:20.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:20.892 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:49:20.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=193273528328 2026-02-17T11:49:20.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 193273528328 2026-02-17T11:49:20.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574868 3-274877906946 4-98784247824 5-115964117007 6-133143986189 7-154618822667 8-176093659146 9-193273528328' 2026-02-17T11:49:20.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:20.964 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836503 2026-02-17T11:49:20.964 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:20.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:49:20.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836503 2026-02-17T11:49:20.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:20.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836503 2026-02-17T11:49:20.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836503' 2026-02-17T11:49:20.968 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836503 2026-02-17T11:49:20.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:49:21.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836502 -lt 21474836503 2026-02-17T11:49:21.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:49:22.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:49:22.134 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:49:22.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836504 -lt 21474836503 2026-02-17T11:49:22.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:22.286 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705686 2026-02-17T11:49:22.286 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:22.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:49:22.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705686 2026-02-17T11:49:22.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:22.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705686 2026-02-17T11:49:22.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705686' 2026-02-17T11:49:22.290 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705686 2026-02-17T11:49:22.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:49:22.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705686 -lt 38654705686 2026-02-17T11:49:22.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:22.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574868 2026-02-17T11:49:22.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:22.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:49:22.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574868 2026-02-17T11:49:22.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:22.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574868 2026-02-17T11:49:22.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574868' 2026-02-17T11:49:22.453 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 55834574868 2026-02-17T11:49:22.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:49:22.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574868 -lt 55834574868 2026-02-17T11:49:22.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:22.611 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-274877906946 2026-02-17T11:49:22.611 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:22.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:49:22.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-274877906946 2026-02-17T11:49:22.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:22.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=274877906946 2026-02-17T11:49:22.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 274877906946' 2026-02-17T11:49:22.615 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.3 seq 274877906946 2026-02-17T11:49:22.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:49:22.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 274877906946 -lt 274877906946 2026-02-17T11:49:22.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:22.767 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247824 2026-02-17T11:49:22.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:22.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:49:22.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247824 2026-02-17T11:49:22.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:22.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247824 2026-02-17T11:49:22.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247824' 2026-02-17T11:49:22.771 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.4 seq 98784247824 2026-02-17T11:49:22.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:49:22.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247825 -lt 98784247824 2026-02-17T11:49:22.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:22.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-115964117007 2026-02-17T11:49:22.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:22.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:49:22.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-115964117007 2026-02-17T11:49:22.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:22.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964117007 2026-02-17T11:49:22.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 115964117007' 2026-02-17T11:49:22.941 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.5 seq 115964117007 2026-02-17T11:49:22.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:49:23.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964117007 -lt 115964117007 2026-02-17T11:49:23.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:23.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986189 2026-02-17T11:49:23.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:23.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:49:23.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986189 2026-02-17T11:49:23.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:23.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986189 2026-02-17T11:49:23.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986189' 2026-02-17T11:49:23.108 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.6 seq 133143986189 2026-02-17T11:49:23.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:49:23.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986189 -lt 133143986189 2026-02-17T11:49:23.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:23.258 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822667 2026-02-17T11:49:23.258 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:23.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:49:23.260 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822667 2026-02-17T11:49:23.260 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:23.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822667 2026-02-17T11:49:23.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822667' 2026-02-17T11:49:23.262 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.7 seq 154618822667 2026-02-17T11:49:23.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:49:23.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822668 -lt 154618822667 2026-02-17T11:49:23.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:23.429 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659146 2026-02-17T11:49:23.429 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:23.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:49:23.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659146 2026-02-17T11:49:23.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:23.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659146 2026-02-17T11:49:23.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659146' 2026-02-17T11:49:23.433 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.8 seq 176093659146 2026-02-17T11:49:23.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:49:23.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659146 -lt 176093659146 2026-02-17T11:49:23.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:23.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-193273528328 2026-02-17T11:49:23.596 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:23.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:49:23.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-193273528328 2026-02-17T11:49:23.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:23.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=193273528328 2026-02-17T11:49:23.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 193273528328' 2026-02-17T11:49:23.599 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.9 seq 193273528328 2026-02-17T11:49:23.599 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:49:23.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528328 -lt 193273528328 2026-02-17T11:49:23.753 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:49:23.753 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:49:23.753 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:49:23.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:49:23.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:49:23.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:49:23.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:49:23.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:49:23.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:49:23.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:49:23.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:49:24.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:49:24.099 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:49:24.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:49:24.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:49:24.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:49:24.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:49:24.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:49:24.286 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-17T11:49:24.286 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:49:24.286 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:49:24.287 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:49:24.287 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:49:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=2.0 2026-02-17T11:49:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 2.0 2026-02-17T11:49:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T11:49:24.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T11:49:24.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:49:24.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:49:24.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:49:24.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:49:24.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:49:06.248039+0000 2026-02-17T11:49:24.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T11:49:24.738 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T11:49:24.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T11:49:06.248039+0000 2026-02-17T11:49:24.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:49:24.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:49:06.248039+0000 2026-02-17T11:49:24.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:49:24.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:49:24.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:49:24.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:49:24.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:49:24.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:49:24.749 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:49:24.749 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:49:24.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:49:06.248039+0000 '>' 2026-02-17T11:49:06.248039+0000 2026-02-17T11:49:24.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:49:25.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:49:25.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:49:25.906 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:49:25.906 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:49:25.907 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:49:25.907 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:49:25.907 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:49:26.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:49:06.248039+0000 '>' 2026-02-17T11:49:06.248039+0000 2026-02-17T11:49:26.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:49:27.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:49:27.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:49:27.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:49:27.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:49:27.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:49:27.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:49:27.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:49:27.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:49:06.248039+0000 '>' 2026-02-17T11:49:06.248039+0000 2026-02-17T11:49:27.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:49:28.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:49:28.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:49:28.220 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:49:28.220 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:49:28.220 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:49:28.221 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:49:28.221 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:49:28.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:49:06.248039+0000 '>' 2026-02-17T11:49:06.248039+0000 2026-02-17T11:49:28.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:49:29.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:49:29.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:49:29.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:49:29.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:49:29.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:49:29.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:49:29.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:49:29.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:49:06.248039+0000 '>' 2026-02-17T11:49:06.248039+0000 2026-02-17T11:49:29.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:49:30.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:49:30.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:49:30.533 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:49:30.533 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:49:30.533 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:49:30.534 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:49:30.534 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:49:30.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:49:25.498859+0000 '>' 2026-02-17T11:49:06.248039+0000 2026-02-17T11:49:30.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:49:30.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:49:30.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:49:30.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:49:30.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T11:49:30.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:49:30.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:49:30.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:49:30.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:49:30.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T11:49:30.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:49:30.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-17T11:49:30.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:49:30.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:49:30.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:49:30.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:49:30.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:49:31.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:49:31.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:49:31.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:49:31.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:49:31.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T11:49:31.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:49:31.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:49:31.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-17T11:49:32.311 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:49:32.312 INFO:tasks.workunit.client.0.trial128.stdout:hinfo_key 2026-02-17T11:49:32.312 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:49:32.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-17T11:49:32.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:49:32.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:49:32.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T11:49:32.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:49:32.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:49:32.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:49:32.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:49:32.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:49:32.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:49:32.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T11:49:32.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T11:49:32.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:49:32.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:49:32.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:49:32.849 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:49:32.849 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:49:32.849 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:49:32.852 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:49:32.852 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:49:32.852 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:49:32.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:49:32.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:49:32.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:49:32.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:49:32.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:49:32.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:49:32.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:49:32.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:49:32.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:49:32.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:49:32.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:49:32.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T11:49:32.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T11:49:32.853 INFO:tasks.workunit.client.0.trial128.stderr:start osd.9 2026-02-17T11:49:32.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:49:32.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-17T11:49:32.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T11:49:32.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:49:32.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:49:32.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:49:32.873 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:32.861+0000 7fb33a2bf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:49:32.874 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:32.865+0000 7fb33a2bf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:49:32.876 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:32.865+0000 7fb33a2bf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:49:33.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T11:49:33.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:49:33.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T11:49:33.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:49:33.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:49:33.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:49:33.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:49:33.005 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:49:33.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:49:33.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:49:33.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:49:34.108 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:34.097+0000 7fb33a2bf8c0 -1 Falling back to public interface 2026-02-17T11:49:34.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:49:34.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:49:34.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:49:34.161 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:49:34.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:49:34.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:49:34.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:49:35.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:49:35.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:49:35.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:49:35.319 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:49:35.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:49:35.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:49:35.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:49:36.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:49:36.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:49:36.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:49:36.477 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:49:36.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:49:36.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:49:36.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:49:37.138 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:37.129+0000 7fb33a2bf8c0 -1 osd.9 65 log_to_monitors true 2026-02-17T11:49:37.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:49:37.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:49:37.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:49:37.635 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:49:37.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:49:37.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:49:37.790 INFO:tasks.workunit.client.0.trial128.stderr:osd.9 up in weight 1 up_from 69 up_thru 0 down_at 66 last_clean_interval [45,65) [v2:127.0.0.1:6874/299165256,v1:127.0.0.1:6875/299165256] [v2:127.0.0.1:6876/299165256,v1:127.0.0.1:6877/299165256] exists,up 6ca155d1-18e1-4577-a926-edebebb3c55d 2026-02-17T11:49:37.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:49:37.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:49:37.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:49:37.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:49:37.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:49:37.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:49:37.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:49:37.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:49:37.792 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:49:37.792 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:49:37.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:49:37.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:49:37.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:49:37.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:49:37.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:49:37.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:49:37.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:49:37.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:49:37.897 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:49:38.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:49:38.051 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:49:38.051 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:49:38.051 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:49:38.052 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:49:38.052 INFO:tasks.workunit.client.0.trial128.stderr:5 2026-02-17T11:49:38.052 INFO:tasks.workunit.client.0.trial128.stderr:6 2026-02-17T11:49:38.052 INFO:tasks.workunit.client.0.trial128.stderr:7 2026-02-17T11:49:38.052 INFO:tasks.workunit.client.0.trial128.stderr:8 2026-02-17T11:49:38.052 INFO:tasks.workunit.client.0.trial128.stderr:9' 2026-02-17T11:49:38.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:49:38.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:38.052 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:49:38.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836508 2026-02-17T11:49:38.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836508 2026-02-17T11:49:38.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508' 2026-02-17T11:49:38.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:38.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:49:38.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705690 2026-02-17T11:49:38.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705690 2026-02-17T11:49:38.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690' 2026-02-17T11:49:38.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:38.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:49:38.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574873 2026-02-17T11:49:38.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574873 2026-02-17T11:49:38.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690 2-55834574873' 2026-02-17T11:49:38.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:38.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:49:38.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=274877906950 2026-02-17T11:49:38.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 274877906950 2026-02-17T11:49:38.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690 2-55834574873 3-274877906950' 2026-02-17T11:49:38.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:38.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:49:38.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247829 2026-02-17T11:49:38.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247829 2026-02-17T11:49:38.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690 2-55834574873 3-274877906950 4-98784247829' 2026-02-17T11:49:38.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:38.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:49:38.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964117011 2026-02-17T11:49:38.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964117011 2026-02-17T11:49:38.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690 2-55834574873 3-274877906950 4-98784247829 5-115964117011' 2026-02-17T11:49:38.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:38.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:49:38.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986194 2026-02-17T11:49:38.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986194 2026-02-17T11:49:38.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690 2-55834574873 3-274877906950 4-98784247829 5-115964117011 6-133143986194' 2026-02-17T11:49:38.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:38.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:49:38.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822672 2026-02-17T11:49:38.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822672 2026-02-17T11:49:38.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690 2-55834574873 3-274877906950 4-98784247829 5-115964117011 6-133143986194 7-154618822672' 2026-02-17T11:49:38.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:38.636 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:49:38.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659150 2026-02-17T11:49:38.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659150 2026-02-17T11:49:38.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690 2-55834574873 3-274877906950 4-98784247829 5-115964117011 6-133143986194 7-154618822672 8-176093659150' 2026-02-17T11:49:38.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:38.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:49:38.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743426 2026-02-17T11:49:38.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743426 2026-02-17T11:49:38.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836508 1-38654705690 2-55834574873 3-274877906950 4-98784247829 5-115964117011 6-133143986194 7-154618822672 8-176093659150 9-296352743426' 2026-02-17T11:49:38.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:38.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836508 2026-02-17T11:49:38.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:38.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:49:38.781 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836508 2026-02-17T11:49:38.781 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:38.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836508 2026-02-17T11:49:38.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836508' 2026-02-17T11:49:38.783 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836508 2026-02-17T11:49:38.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:49:38.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836508 -lt 21474836508 2026-02-17T11:49:38.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:38.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705690 2026-02-17T11:49:38.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:38.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:49:38.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705690 2026-02-17T11:49:38.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:38.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705690 2026-02-17T11:49:38.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705690' 2026-02-17T11:49:38.943 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705690 2026-02-17T11:49:38.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:49:39.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705690 -lt 38654705690 2026-02-17T11:49:39.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:39.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574873 2026-02-17T11:49:39.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:39.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:49:39.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574873 2026-02-17T11:49:39.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:39.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574873 2026-02-17T11:49:39.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574873' 2026-02-17T11:49:39.099 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 55834574873 2026-02-17T11:49:39.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:49:39.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574872 -lt 55834574873 2026-02-17T11:49:39.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:49:40.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:49:40.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:49:40.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574873 -lt 55834574873 2026-02-17T11:49:40.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:40.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-274877906950 2026-02-17T11:49:40.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:40.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:49:40.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-274877906950 2026-02-17T11:49:40.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:40.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=274877906950 2026-02-17T11:49:40.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 274877906950' 2026-02-17T11:49:40.412 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.3 seq 274877906950 2026-02-17T11:49:40.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:49:40.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 274877906951 -lt 274877906950 2026-02-17T11:49:40.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:40.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247829 2026-02-17T11:49:40.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:40.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:49:40.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247829 2026-02-17T11:49:40.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:40.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247829 2026-02-17T11:49:40.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247829' 2026-02-17T11:49:40.582 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.4 seq 98784247829 2026-02-17T11:49:40.582 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:49:40.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247829 -lt 98784247829 2026-02-17T11:49:40.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:40.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-115964117011 2026-02-17T11:49:40.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:40.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:49:40.746 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-115964117011 2026-02-17T11:49:40.746 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:40.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964117011 2026-02-17T11:49:40.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 115964117011' 2026-02-17T11:49:40.748 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.5 seq 115964117011 2026-02-17T11:49:40.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:49:40.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964117012 -lt 115964117011 2026-02-17T11:49:40.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:40.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986194 2026-02-17T11:49:40.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:40.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:49:40.907 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986194 2026-02-17T11:49:40.907 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:40.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986194 2026-02-17T11:49:40.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986194' 2026-02-17T11:49:40.908 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.6 seq 133143986194 2026-02-17T11:49:40.909 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:49:41.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986194 -lt 133143986194 2026-02-17T11:49:41.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:41.058 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822672 2026-02-17T11:49:41.058 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:41.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:49:41.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822672 2026-02-17T11:49:41.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:41.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822672 2026-02-17T11:49:41.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822672' 2026-02-17T11:49:41.062 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.7 seq 154618822672 2026-02-17T11:49:41.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:49:41.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822672 -lt 154618822672 2026-02-17T11:49:41.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:41.220 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659150 2026-02-17T11:49:41.220 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:41.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:49:41.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659150 2026-02-17T11:49:41.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:41.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659150 2026-02-17T11:49:41.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659150' 2026-02-17T11:49:41.224 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.8 seq 176093659150 2026-02-17T11:49:41.224 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:49:41.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659150 -lt 176093659150 2026-02-17T11:49:41.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:41.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-296352743426 2026-02-17T11:49:41.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:41.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:49:41.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-296352743426 2026-02-17T11:49:41.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:41.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743426 2026-02-17T11:49:41.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 296352743426' 2026-02-17T11:49:41.387 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.9 seq 296352743426 2026-02-17T11:49:41.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:49:41.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743426 -lt 296352743426 2026-02-17T11:49:41.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:49:41.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:49:41.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:49:41.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:49:41.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:49:41.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:49:41.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:49:41.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:49:41.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:49:41.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:49:41.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:49:41.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:49:41.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:49:41.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:49:41.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:49:42.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:49:42.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:49:42.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:49:42.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:49:42.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:49:42.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:261: corrupt_and_repair_erasure_coded: corrupt_and_repair_one td/osd-scrub-repair ecpool 5 2026-02-17T11:49:42.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T11:49:42.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=ecpool 2026-02-17T11:49:42.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=5 2026-02-17T11:49:42.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:49:42.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:49:42.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:49:42.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T11:49:42.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:49:42.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:49:42.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:49:42.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:49:42.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T11:49:42.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:49:42.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T11:49:42.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:49:42.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:49:42.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:49:42.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:49:42.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:49:43.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:49:43.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:49:43.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:49:43.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:49:43.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T11:49:43.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:49:43.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:49:43.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-17T11:49:44.051 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T11:49:44.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T11:49:44.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:49:44.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:49:44.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T11:49:44.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:49:44.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:49:44.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:49:44.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:49:44.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:49:44.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:49:44.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T11:49:44.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T11:49:44.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:49:44.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:49:44.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:49:44.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:49:44.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:49:44.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:49:44.589 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:49:44.589 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:49:44.589 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:49:44.590 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:49:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:49:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:49:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:49:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:49:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:49:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:49:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:49:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:49:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:49:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:49:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T11:49:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T11:49:44.593 INFO:tasks.workunit.client.0.trial128.stderr:start osd.5 2026-02-17T11:49:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:49:44.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T11:49:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T11:49:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:49:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:49:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:49:44.604 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:44.593+0000 7f57b6ad48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:49:44.605 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:44.593+0000 7f57b6ad48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:49:44.606 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:44.597+0000 7f57b6ad48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:49:44.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T11:49:44.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:49:44.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T11:49:44.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:49:44.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:49:44.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:49:44.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:49:44.747 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:49:44.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:49:44.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:49:44.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:49:45.867 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:45.857+0000 7f57b6ad48c0 -1 Falling back to public interface 2026-02-17T11:49:45.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:49:45.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:49:45.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:49:45.901 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:49:45.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:49:45.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:49:46.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:49:47.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:49:47.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:49:47.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:49:47.053 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:49:47.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:49:47.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:49:47.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:49:48.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:49:48.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:49:48.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:49:48.209 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:49:48.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:49:48.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:49:48.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:49:49.120 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:49.109+0000 7f57b6ad48c0 -1 osd.5 71 log_to_monitors true 2026-02-17T11:49:49.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:49:49.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:49:49.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:49:49.361 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:49:49.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:49:49.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:49:49.514 INFO:tasks.workunit.client.0.trial128.stderr:osd.5 up in weight 1 up_from 76 up_thru 76 down_at 72 last_clean_interval [27,71) [v2:127.0.0.1:6842/203460971,v1:127.0.0.1:6843/203460971] [v2:127.0.0.1:6844/203460971,v1:127.0.0.1:6845/203460971] exists,up 80987ebb-2096-4a73-b7c5-64048030732e 2026-02-17T11:49:49.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:49:49.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:49:49.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:49:49.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:49:49.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:49:49.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:49:49.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:49:49.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:49:49.516 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:49:49.516 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:49:49.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:49:49.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:49:49.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:49:49.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:49:49.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:49:49.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:49:49.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:49:49.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:49:49.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:49:49.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:49:49.784 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:49:49.784 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:49:49.785 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:49:49.785 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:49:49.785 INFO:tasks.workunit.client.0.trial128.stderr:5 2026-02-17T11:49:49.785 INFO:tasks.workunit.client.0.trial128.stderr:6 2026-02-17T11:49:49.785 INFO:tasks.workunit.client.0.trial128.stderr:7 2026-02-17T11:49:49.785 INFO:tasks.workunit.client.0.trial128.stderr:8 2026-02-17T11:49:49.785 INFO:tasks.workunit.client.0.trial128.stderr:9' 2026-02-17T11:49:49.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:49:49.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:49.786 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:49:49.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836511 2026-02-17T11:49:49.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836511 2026-02-17T11:49:49.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511' 2026-02-17T11:49:49.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:49.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:49:49.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705694 2026-02-17T11:49:49.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705694 2026-02-17T11:49:49.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694' 2026-02-17T11:49:49.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:49.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:49:49.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574876 2026-02-17T11:49:50.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574876 2026-02-17T11:49:50.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694 2-55834574876' 2026-02-17T11:49:50.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:50.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:49:50.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=274877906954 2026-02-17T11:49:50.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 274877906954 2026-02-17T11:49:50.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694 2-55834574876 3-274877906954' 2026-02-17T11:49:50.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:50.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:49:50.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247832 2026-02-17T11:49:50.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247832 2026-02-17T11:49:50.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694 2-55834574876 3-274877906954 4-98784247832' 2026-02-17T11:49:50.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:50.144 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:49:50.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=326417514498 2026-02-17T11:49:50.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 326417514498 2026-02-17T11:49:50.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694 2-55834574876 3-274877906954 4-98784247832 5-326417514498' 2026-02-17T11:49:50.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:50.215 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:49:50.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986197 2026-02-17T11:49:50.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986197 2026-02-17T11:49:50.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694 2-55834574876 3-274877906954 4-98784247832 5-326417514498 6-133143986197' 2026-02-17T11:49:50.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:50.284 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:49:50.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822675 2026-02-17T11:49:50.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822675 2026-02-17T11:49:50.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694 2-55834574876 3-274877906954 4-98784247832 5-326417514498 6-133143986197 7-154618822675' 2026-02-17T11:49:50.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:50.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:49:50.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659153 2026-02-17T11:49:50.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659153 2026-02-17T11:49:50.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694 2-55834574876 3-274877906954 4-98784247832 5-326417514498 6-133143986197 7-154618822675 8-176093659153' 2026-02-17T11:49:50.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:49:50.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:49:50.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=296352743429 2026-02-17T11:49:50.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 296352743429 2026-02-17T11:49:50.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836511 1-38654705694 2-55834574876 3-274877906954 4-98784247832 5-326417514498 6-133143986197 7-154618822675 8-176093659153 9-296352743429' 2026-02-17T11:49:50.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:50.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836511 2026-02-17T11:49:50.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:50.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:49:50.498 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836511 2026-02-17T11:49:50.498 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:50.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836511 2026-02-17T11:49:50.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836511' 2026-02-17T11:49:50.500 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836511 2026-02-17T11:49:50.500 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:49:50.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836511 -lt 21474836511 2026-02-17T11:49:50.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:50.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705694 2026-02-17T11:49:50.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:50.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:49:50.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705694 2026-02-17T11:49:50.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:50.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705694 2026-02-17T11:49:50.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705694' 2026-02-17T11:49:50.653 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705694 2026-02-17T11:49:50.653 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:49:50.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705694 -lt 38654705694 2026-02-17T11:49:50.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:50.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574876 2026-02-17T11:49:50.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:50.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:49:50.819 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574876 2026-02-17T11:49:50.819 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:50.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574876 2026-02-17T11:49:50.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574876' 2026-02-17T11:49:50.820 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 55834574876 2026-02-17T11:49:50.821 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:49:50.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574876 -lt 55834574876 2026-02-17T11:49:50.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:50.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-274877906954 2026-02-17T11:49:50.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:50.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:49:50.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-274877906954 2026-02-17T11:49:50.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:50.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=274877906954 2026-02-17T11:49:50.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 274877906954' 2026-02-17T11:49:50.979 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.3 seq 274877906954 2026-02-17T11:49:50.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:49:51.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 274877906954 -lt 274877906954 2026-02-17T11:49:51.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:51.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247832 2026-02-17T11:49:51.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:51.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:49:51.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247832 2026-02-17T11:49:51.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:51.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247832 2026-02-17T11:49:51.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247832' 2026-02-17T11:49:51.137 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.4 seq 98784247832 2026-02-17T11:49:51.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:49:51.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247832 -lt 98784247832 2026-02-17T11:49:51.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:51.303 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-326417514498 2026-02-17T11:49:51.303 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:51.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:49:51.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-326417514498 2026-02-17T11:49:51.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:51.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=326417514498 2026-02-17T11:49:51.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 326417514498' 2026-02-17T11:49:51.307 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.5 seq 326417514498 2026-02-17T11:49:51.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:49:51.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 326417514498 -lt 326417514498 2026-02-17T11:49:51.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:51.463 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986197 2026-02-17T11:49:51.463 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:51.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:49:51.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986197 2026-02-17T11:49:51.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:51.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986197 2026-02-17T11:49:51.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986197' 2026-02-17T11:49:51.467 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.6 seq 133143986197 2026-02-17T11:49:51.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:49:51.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986196 -lt 133143986197 2026-02-17T11:49:51.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:49:52.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:49:52.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:49:52.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986197 -lt 133143986197 2026-02-17T11:49:52.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:52.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822675 2026-02-17T11:49:52.790 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:52.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:49:52.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822675 2026-02-17T11:49:52.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:52.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822675 2026-02-17T11:49:52.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822675' 2026-02-17T11:49:52.793 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.7 seq 154618822675 2026-02-17T11:49:52.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:49:52.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822676 -lt 154618822675 2026-02-17T11:49:52.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:52.946 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659153 2026-02-17T11:49:52.946 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:52.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:49:52.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659153 2026-02-17T11:49:52.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:52.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659153 2026-02-17T11:49:52.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659153' 2026-02-17T11:49:52.950 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.8 seq 176093659153 2026-02-17T11:49:52.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:49:53.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659154 -lt 176093659153 2026-02-17T11:49:53.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:49:53.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-296352743429 2026-02-17T11:49:53.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:49:53.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:49:53.110 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-296352743429 2026-02-17T11:49:53.110 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:49:53.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=296352743429 2026-02-17T11:49:53.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 296352743429' 2026-02-17T11:49:53.112 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.9 seq 296352743429 2026-02-17T11:49:53.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:49:53.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 296352743430 -lt 296352743429 2026-02-17T11:49:53.274 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:49:53.274 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:49:53.274 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:49:53.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:49:53.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:49:53.456 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:49:53.456 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:49:53.457 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:49:53.457 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:49:53.457 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:49:53.457 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:49:53.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:49:53.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:49:53.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:49:53.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:49:53.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:49:53.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:49:53.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:49:53.786 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg ecpool SOMETHING 2026-02-17T11:49:53.786 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:49:53.786 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:49:53.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:49:53.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:49:53.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=2.0 2026-02-17T11:49:53.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 2.0 2026-02-17T11:49:53.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T11:49:53.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T11:49:53.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:49:53.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:49:53.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:49:53.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:49:54.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:49:25.498859+0000 2026-02-17T11:49:54.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T11:49:54.235 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T11:49:54.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T11:49:25.498859+0000 2026-02-17T11:49:54.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:49:54.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:49:25.498859+0000 2026-02-17T11:49:54.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:49:54.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:49:54.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:49:54.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:49:54.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:49:54.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:49:54.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:49:54.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:49:54.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:49:25.498859+0000 '>' 2026-02-17T11:49:25.498859+0000 2026-02-17T11:49:54.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:49:55.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:49:55.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:49:55.399 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:49:55.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:49:55.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:49:55.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:49:55.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:49:55.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:49:25.498859+0000 '>' 2026-02-17T11:49:25.498859+0000 2026-02-17T11:49:55.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:49:56.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:49:56.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:49:56.554 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:49:56.554 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:49:56.554 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:49:56.555 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:49:56.555 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:49:56.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:49:54.371607+0000 '>' 2026-02-17T11:49:25.498859+0000 2026-02-17T11:49:56.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:49:56.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:49:56.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:49:56.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:49:56.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T11:49:56.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:49:56.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:49:56.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:49:56.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:49:56.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T11:49:56.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:49:56.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-17T11:49:56.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:49:56.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:49:56.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:49:56.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:49:56.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:49:58.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:49:58.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:49:58.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:49:58.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:49:58.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T11:49:58.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:49:58.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:49:58.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-17T11:49:58.336 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:49:58.336 INFO:tasks.workunit.client.0.trial128.stdout:hinfo_key 2026-02-17T11:49:58.337 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:49:58.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-17T11:49:58.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:49:58.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:49:58.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T11:49:58.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:49:58.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:49:58.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:49:58.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:49:58.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:49:58.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:49:58.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T11:49:58.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T11:49:58.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:49:58.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:49:58.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:49:58.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:49:58.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:49:58.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:49:58.874 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:49:58.874 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:49:58.874 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:49:58.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:49:58.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:49:58.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:49:58.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:49:58.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:49:58.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:49:58.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:49:58.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:49:58.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:49:58.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:49:58.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:49:58.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T11:49:58.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T11:49:58.877 INFO:tasks.workunit.client.0.trial128.stderr:start osd.9 2026-02-17T11:49:58.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:49:58.877 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-17T11:49:58.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T11:49:58.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:49:58.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:49:58.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:49:58.891 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:58.880+0000 7f1c059da8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:49:58.893 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:58.880+0000 7f1c059da8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:49:58.894 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:49:58.884+0000 7f1c059da8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:49:59.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T11:49:59.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:49:59.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T11:49:59.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:49:59.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:49:59.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:49:59.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:49:59.035 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:49:59.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:49:59.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:49:59.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:00.115 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:50:00.104+0000 7f1c059da8c0 -1 Falling back to public interface 2026-02-17T11:50:00.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:00.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:00.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:50:00.188 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:50:00.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:00.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:50:00.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:01.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:01.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:01.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:50:01.336 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:50:01.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:01.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:50:01.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:02.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:02.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:02.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:50:02.487 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:50:02.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:02.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:50:02.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:03.158 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:50:03.148+0000 7f1c059da8c0 -1 osd.9 79 log_to_monitors true 2026-02-17T11:50:03.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:03.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:03.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:50:03.636 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:50:03.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:03.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:50:03.794 INFO:tasks.workunit.client.0.trial128.stderr:osd.9 up in weight 1 up_from 83 up_thru 0 down_at 80 last_clean_interval [69,79) [v2:127.0.0.1:6874/3403569106,v1:127.0.0.1:6875/3403569106] [v2:127.0.0.1:6876/3403569106,v1:127.0.0.1:6877/3403569106] exists,up 6ca155d1-18e1-4577-a926-edebebb3c55d 2026-02-17T11:50:03.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:50:03.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:50:03.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:50:03.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:50:03.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:50:03.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:50:03.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:50:03.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:50:03.795 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:50:03.795 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:50:03.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:50:03.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:50:03.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:50:03.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:50:03.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:50:03.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:50:03.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:50:03.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:50:03.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:50:04.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:50:04.056 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:50:04.056 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:50:04.056 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:50:04.056 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:50:04.057 INFO:tasks.workunit.client.0.trial128.stderr:5 2026-02-17T11:50:04.057 INFO:tasks.workunit.client.0.trial128.stderr:6 2026-02-17T11:50:04.057 INFO:tasks.workunit.client.0.trial128.stderr:7 2026-02-17T11:50:04.057 INFO:tasks.workunit.client.0.trial128.stderr:8 2026-02-17T11:50:04.057 INFO:tasks.workunit.client.0.trial128.stderr:9' 2026-02-17T11:50:04.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:50:04.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:04.057 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:50:04.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836515 2026-02-17T11:50:04.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836515 2026-02-17T11:50:04.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515' 2026-02-17T11:50:04.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:04.128 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:50:04.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705697 2026-02-17T11:50:04.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705697 2026-02-17T11:50:04.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697' 2026-02-17T11:50:04.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:04.201 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:50:04.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574880 2026-02-17T11:50:04.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574880 2026-02-17T11:50:04.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-55834574880' 2026-02-17T11:50:04.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:04.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:50:04.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=274877906957 2026-02-17T11:50:04.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 274877906957 2026-02-17T11:50:04.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-55834574880 3-274877906957' 2026-02-17T11:50:04.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:04.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:50:04.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247836 2026-02-17T11:50:04.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247836 2026-02-17T11:50:04.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-55834574880 3-274877906957 4-98784247836' 2026-02-17T11:50:04.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:04.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:50:04.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=326417514502 2026-02-17T11:50:04.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 326417514502 2026-02-17T11:50:04.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-55834574880 3-274877906957 4-98784247836 5-326417514502' 2026-02-17T11:50:04.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:04.489 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:50:04.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986201 2026-02-17T11:50:04.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986201 2026-02-17T11:50:04.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-55834574880 3-274877906957 4-98784247836 5-326417514502 6-133143986201' 2026-02-17T11:50:04.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:04.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:50:04.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822679 2026-02-17T11:50:04.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822679 2026-02-17T11:50:04.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-55834574880 3-274877906957 4-98784247836 5-326417514502 6-133143986201 7-154618822679' 2026-02-17T11:50:04.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:04.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:50:04.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659157 2026-02-17T11:50:04.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659157 2026-02-17T11:50:04.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-55834574880 3-274877906957 4-98784247836 5-326417514502 6-133143986201 7-154618822679 8-176093659157' 2026-02-17T11:50:04.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:04.703 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:50:04.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=356482285570 2026-02-17T11:50:04.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 356482285570 2026-02-17T11:50:04.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836515 1-38654705697 2-55834574880 3-274877906957 4-98784247836 5-326417514502 6-133143986201 7-154618822679 8-176093659157 9-356482285570' 2026-02-17T11:50:04.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:04.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836515 2026-02-17T11:50:04.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:04.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:50:04.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836515 2026-02-17T11:50:04.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:04.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836515 2026-02-17T11:50:04.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836515' 2026-02-17T11:50:04.780 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836515 2026-02-17T11:50:04.781 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:50:04.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836515 -lt 21474836515 2026-02-17T11:50:04.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:04.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705697 2026-02-17T11:50:04.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:04.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:50:04.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705697 2026-02-17T11:50:04.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:04.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705697 2026-02-17T11:50:04.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705697' 2026-02-17T11:50:04.940 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705697 2026-02-17T11:50:04.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:50:05.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705697 -lt 38654705697 2026-02-17T11:50:05.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:05.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574880 2026-02-17T11:50:05.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:05.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:50:05.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574880 2026-02-17T11:50:05.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:05.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574880 2026-02-17T11:50:05.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574880' 2026-02-17T11:50:05.102 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 55834574880 2026-02-17T11:50:05.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:50:05.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574879 -lt 55834574880 2026-02-17T11:50:05.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:50:06.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:50:06.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:50:06.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574880 -lt 55834574880 2026-02-17T11:50:06.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:06.411 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-274877906957 2026-02-17T11:50:06.411 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:06.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:50:06.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-274877906957 2026-02-17T11:50:06.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:06.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=274877906957 2026-02-17T11:50:06.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 274877906957' 2026-02-17T11:50:06.414 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.3 seq 274877906957 2026-02-17T11:50:06.415 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:50:06.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 274877906958 -lt 274877906957 2026-02-17T11:50:06.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:06.566 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247836 2026-02-17T11:50:06.566 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:06.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:50:06.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247836 2026-02-17T11:50:06.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:06.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247836 2026-02-17T11:50:06.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247836' 2026-02-17T11:50:06.570 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.4 seq 98784247836 2026-02-17T11:50:06.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:50:06.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247837 -lt 98784247836 2026-02-17T11:50:06.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:06.726 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-326417514502 2026-02-17T11:50:06.726 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:06.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:50:06.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-326417514502 2026-02-17T11:50:06.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:06.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=326417514502 2026-02-17T11:50:06.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 326417514502' 2026-02-17T11:50:06.730 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.5 seq 326417514502 2026-02-17T11:50:06.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:50:06.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 326417514502 -lt 326417514502 2026-02-17T11:50:06.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:06.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986201 2026-02-17T11:50:06.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:06.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:50:06.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986201 2026-02-17T11:50:06.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:06.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986201 2026-02-17T11:50:06.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986201' 2026-02-17T11:50:06.889 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.6 seq 133143986201 2026-02-17T11:50:06.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:50:07.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986201 -lt 133143986201 2026-02-17T11:50:07.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:07.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822679 2026-02-17T11:50:07.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:07.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:50:07.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822679 2026-02-17T11:50:07.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:07.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822679 2026-02-17T11:50:07.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822679' 2026-02-17T11:50:07.054 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.7 seq 154618822679 2026-02-17T11:50:07.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:50:07.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822679 -lt 154618822679 2026-02-17T11:50:07.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:07.205 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659157 2026-02-17T11:50:07.205 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:07.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:50:07.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659157 2026-02-17T11:50:07.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:07.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659157 2026-02-17T11:50:07.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659157' 2026-02-17T11:50:07.209 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.8 seq 176093659157 2026-02-17T11:50:07.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:50:07.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659158 -lt 176093659157 2026-02-17T11:50:07.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:07.377 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-356482285570 2026-02-17T11:50:07.377 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:07.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:50:07.379 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-356482285570 2026-02-17T11:50:07.379 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:07.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=356482285570 2026-02-17T11:50:07.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 356482285570' 2026-02-17T11:50:07.382 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.9 seq 356482285570 2026-02-17T11:50:07.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:50:07.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 356482285570 -lt 356482285570 2026-02-17T11:50:07.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:50:07.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:50:07.547 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:50:07.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:50:07.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:50:07.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:50:07.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:50:07.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:50:07.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:50:07.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:50:07.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:50:07.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:50:07.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:50:07.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:50:07.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:50:08.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:50:08.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:50:08.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:50:08.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:50:08.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:50:08.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:262: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 5 9 2026-02-17T11:50:08.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:185: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-17T11:50:08.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:186: corrupt_and_repair_two: local poolname=ecpool 2026-02-17T11:50:08.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: corrupt_and_repair_two: local first=5 2026-02-17T11:50:08.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: corrupt_and_repair_two: local second=9 2026-02-17T11:50:08.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:193: corrupt_and_repair_two: pids= 2026-02-17T11:50:08.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:50:08.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:50:08.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:50:08.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 232629"' 2026-02-17T11:50:08.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 232629' 2026-02-17T11:50:08.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 9 SOMETHING remove 2026-02-17T11:50:08.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:50:08.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:50:08.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 232631"' 2026-02-17T11:50:08.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:50:08.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 232631' 2026-02-17T11:50:08.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: corrupt_and_repair_two: wait_background pids 2026-02-17T11:50:08.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 232629 232631' 2026-02-17T11:50:08.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T11:50:08.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:50:08.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 232629 2026-02-17T11:50:08.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/232632: /' 2026-02-17T11:50:08.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 9 SOMETHING remove 2026-02-17T11:50:08.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/232634: /' 2026-02-17T11:50:10.602 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:50:10.602 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:50:10.602 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T11:50:10.602 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:50:10.602 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING remove 2026-02-17T11:50:10.602 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:50:10.603 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:50:10.603 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T11:50:10.603 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:50:10.603 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-17T11:50:10.603 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:50:10.603 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:50:10.603 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:50:10.603 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:50:10.603 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:50:10.603 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:50:10.603 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING remove 2026-02-17T11:50:10.603 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:50:10.604 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:50:10.604 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T11:50:10.604 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:50:10.604 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:50:10.604 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING remove 2026-02-17T11:50:10.604 INFO:tasks.workunit.client.0.trial128.stderr:232634: remove 2#2:eb822e21:::SOMETHING:head# 2026-02-17T11:50:10.604 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-17T11:50:10.604 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:50:10.604 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:50:10.604 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T11:50:10.604 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:50:10.604 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:50:10.605 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:50:10.605 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:50:10.605 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:50:10.605 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:50:10.605 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T11:50:10.605 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T11:50:10.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:50:10.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:50:10.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:50:10.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:50:10.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:50:10.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:50:10.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:50:10.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:50:10.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:50:10.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:50:10.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:50:10.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:50:10.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:50:10.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:50:10.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:50:10.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:50:10.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:50:10.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:50:10.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:50:10.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:50:10.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T11:50:10.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T11:50:10.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: start osd.9 2026-02-17T11:50:10.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:50:10.641 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:50:10.641 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:50:10.641 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T11:50:10.641 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:50:10.641 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:50:10.642 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:50:10.642 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:50:10.642 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T11:50:10.642 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:50:10.642 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T11:50:10.642 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:50:10.642 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:50:10.642 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:50:10.642 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:50:10.642 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:50:10.642 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:50:10.642 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:50:10.643 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:50:10.643 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:50:10.643 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T11:50:10.643 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:50:10.643 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:50:10.643 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-17T11:50:10.643 INFO:tasks.workunit.client.0.trial128.stderr:232632: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T11:50:10.643 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T11:50:10.643 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:50:10.643 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:50:10.643 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T11:50:10.643 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:50:10.643 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:50:10.644 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:50:10.644 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:50:10.644 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:50:10.645 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:50:10.645 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T11:50:10.645 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T11:50:10.645 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:50:10.645 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:50:10.645 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:50:10.645 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:50:10.645 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:50:10.645 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:50:10.645 INFO:tasks.workunit.client.0.trial128.stderr:232632: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:50:10.645 INFO:tasks.workunit.client.0.trial128.stderr:232632: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:50:10.646 INFO:tasks.workunit.client.0.trial128.stderr:232632: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:50:10.646 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:50:10.646 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:50:10.646 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:50:10.646 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:50:10.646 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:50:10.646 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:50:10.646 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:50:10.646 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:50:10.646 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:50:10.646 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:50:10.646 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:50:10.647 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T11:50:10.647 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T11:50:10.647 INFO:tasks.workunit.client.0.trial128.stderr:232632: start osd.5 2026-02-17T11:50:10.647 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:50:14.915 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-17T11:50:14.915 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T11:50:14.915 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:50:14.915 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:50:14.915 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:50:14.915 INFO:tasks.workunit.client.0.trial128.stderr:232634: 2026-02-17T11:50:10.612+0000 7fb4bdd888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:14.915 INFO:tasks.workunit.client.0.trial128.stderr:232634: 2026-02-17T11:50:10.612+0000 7fb4bdd888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:14.915 INFO:tasks.workunit.client.0.trial128.stderr:232634: 2026-02-17T11:50:10.616+0000 7fb4bdd888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: 0 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: 2026-02-17T11:50:11.860+0000 7fb4bdd888c0 -1 Falling back to public interface 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: 1 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:14.916 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:14.917 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:50:14.917 INFO:tasks.workunit.client.0.trial128.stderr:232634: 2 2026-02-17T11:50:14.917 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:14.917 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:50:14.917 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:14.917 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:14.917 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:14.917 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:50:14.917 INFO:tasks.workunit.client.0.trial128.stderr:232634: 3 2026-02-17T11:50:14.917 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:14.917 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:50:14.917 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:15.251 INFO:tasks.workunit.client.0.trial128.stderr:232634: 2026-02-17T11:50:14.904+0000 7fb4bdd888c0 232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T11:50:15.251 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T11:50:15.251 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:50:15.251 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: 2026-02-17T11:50:10.648+0000 7f5a79bdd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: 2026-02-17T11:50:10.648+0000 7f5a79bdd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: 2026-02-17T11:50:10.648+0000 7f5a79bdd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: 0 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:15.252 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: 1 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: 2026-02-17T11:50:12.172+0000 7f5a79bdd8c0 -1 Falling back to public interface 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: 2 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: 3 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:15.253 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:50:15.254 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:15.885 INFO:tasks.workunit.client.0.trial128.stderr:232632: 2026-02-17T11:50:15.240+0000 7f5a79bdd8c0 -1 osd.9 84 log_to_monitors true 2026-02-17T11:50:15.885 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:15.885 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:15.885 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:50:15.885 INFO:tasks.workunit.client.0.trial128.stderr:232634: 4 2026-02-17T11:50:15.885 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:15.885 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:50:15.885 INFO:tasks.workunit.client.0.trial128.stderr:232634: osd.9 up in weight 1 up_from 89 up_thru 0 down_at 85 last_clean_interval [83,84) [v2:127.0.0.1:6842/4122241563,v1:127.0.0.1:6843/4122241563] [v2:127.0.0.1:6844/4122241563,v1:127.0.0.1:6845/4122241563] exists,up 6ca155d1-18e1-4577-a926-edebebb3c55d 2026-02-17T11:50:15.885 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:50:15.886 INFO:tasks.workunit.client.0.trial128.stderr:232634: 1 2026-02-17T11:50:15.887 INFO:tasks.workunit.client.0.trial128.stderr:232634: 2 2026-02-17T11:50:15.887 INFO:tasks.workunit.client.0.trial128.stderr:232634: 3 2026-02-17T11:50:15.887 INFO:tasks.workunit.client.0.trial128.stderr:232634: 4 2026-02-17T11:50:15.887 INFO:tasks.workunit.client.0.trial128.stderr:232634: 5 2026-02-17T11:50:15.887 INFO:tasks.workunit.client.0.trial128.stderr:232634: 6 2026-02-17T11:50:15.887 INFO:tasks.workunit.client.0.trial128.stderr:232634: 7 2026-02-17T11:50:15.887 INFO:tasks.workunit.client.0.trial128.stderr:232634: 8 2026-02-17T11:50:15.887 INFO:tasks.workunit.client.0.trial128.stderr:232634: 9' 2026-02-17T11:50:15.887 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:50:15.887 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:15.887 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:50:15.887 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836519 2026-02-17T11:50:15.887 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836519 2026-02-17T11:50:15.887 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519' 2026-02-17T11:50:15.887 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:15.893 INFO:tasks.workunit.client.0.trial128.stderr:232634: /-1 osd.5 84 log_to_monitors true 2026-02-17T11:50:15.893 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:15.893 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:15.893 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:50:15.893 INFO:tasks.workunit.client.0.trial128.stderr:232632: 4 2026-02-17T11:50:15.893 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:15.893 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:50:15.893 INFO:tasks.workunit.client.0.trial128.stderr:232632: osd.5 up in weight 1 up_from 92 up_thru 92 down_at 85 last_clean_interval [76,84) [v2:127.0.0.1:6874/2165343191,v1:127.0.0.1:6875/2165343191] [v2:127.0.0.1:6876/2165343191,v1:127.0.0.1:6877/2165343191] exists,up 80987ebb-2096-4a73-b7c5-64048030732e 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:50:15.894 INFO:tasks.workunit.client.0.trial128.stderr:232632: 1 2026-02-17T11:50:15.895 INFO:tasks.workunit.client.0.trial128.stderr:232632: 2 2026-02-17T11:50:15.895 INFO:tasks.workunit.client.0.trial128.stderr:232632: 3 2026-02-17T11:50:15.895 INFO:tasks.workunit.client.0.trial128.stderr:232632: 4 2026-02-17T11:50:15.895 INFO:tasks.workunit.client.0.trial128.stderr:232632: 5 2026-02-17T11:50:15.895 INFO:tasks.workunit.client.0.trial128.stderr:232632: 6 2026-02-17T11:50:15.895 INFO:tasks.workunit.client.0.trial128.stderr:232632: 7 2026-02-17T11:50:15.895 INFO:tasks.workunit.client.0.trial128.stderr:232632: 8 2026-02-17T11:50:15.895 INFO:tasks.workunit.client.0.trial128.stderr:232632: 9' 2026-02-17T11:50:15.895 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:50:15.895 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:15.895 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:50:15.895 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836520 2026-02-17T11:50:15.895 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836520 2026-02-17T11:50:15.895 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520' 2026-02-17T11:50:15.895 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705701 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705701 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701' 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574883 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574883 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-55834574883' 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=274877906961 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 274877906961 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-55834574883 3-274877906961' 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247840 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247840 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-55834574883 3-274877906961 4-98784247840' 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=395136991234 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 395136991234 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-55834574883 3-274877906961 4-98784247840 5-395136991234' 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986204 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986204 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-55834574883 3-274877906961 4-98784247840 5-395136991234 6-133143986204' 2026-02-17T11:50:16.389 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.390 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:50:16.400 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_st//home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:50:16.400 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705702 2026-02-17T11:50:16.400 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705702 2026-02-17T11:50:16.400 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702' 2026-02-17T11:50:16.400 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.400 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:50:16.400 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574884 2026-02-17T11:50:16.400 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574884 2026-02-17T11:50:16.400 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702 2-55834574884' 2026-02-17T11:50:16.400 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.400 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=274877906962 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 274877906962 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702 2-55834574884 3-274877906962' 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247841 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247841 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702 2-55834574884 3-274877906962 4-98784247841' 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=395136991235 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 395136991235 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702 2-55834574884 3-274877906962 4-98784247841 5-395136991235' 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986205 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986205 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702 2-55834574884 3-274877906962 4-98784247841 5-395136991235 6-133143986205' 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.401 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:50:16.698 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822683 2026-02-17T11:50:16.698 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822683 2026-02-17T11:50:16.698 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702 2-55834574884 3-274877906962 4-98784247841 5-395136991235 6-133143986205 7-154618822683' 2026-02-17T11:50:16.698 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.698 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:50:16.699 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659162 2026-02-17T11:50:16.699 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659162 2026-02-17T11:50:16.699 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702 2-55834574884 3-274877906962 4-98784247841 5-395136991235 6-133143986205 7-154618822683 8-176093659162' 2026-02-17T11:50:16.699 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.699 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:50:16.699 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=382252089347 2026-02-17T11:50:16.699 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 382252089347 2026-02-17T11:50:16.699 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836520 1-38654705702 2-55834574884 3-274877906962 4-98784247841 5-395136991235 6-133143986205 7-154618822683 8-176093659162 9-382252089347' 2026-02-17T11:50:16.699 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:16.699 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836520 2026-02-17T11:50:16.699 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:16.700 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:50:16.700 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836520 2026-02-17T11:50:16.700 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:16.700 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836520 2026-02-17T11:50:16.700 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836520' 2026-02-17T11:50:16.700 INFO:tasks.workunit.client.0.trial128.stderr:232632: waiting osd.0 seq 21474836520 2026-02-17T11:50:16.700 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:50:16.700 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836520 -lt 21474836520 2026-02-17T11:50:16.700 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:16.700 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705702 2026-02-17T11:50:16.700 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:16.700 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:50:16.700 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705702 2026-02-17T11:50:16.701 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:16.701 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705702 2026-02-17T11:50:16.701 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705702' 2026-02-17T11:50:16.701 INFO:tasks.workunit.client.0.trial128.stderr:232632: waiting osd.1 seq 38654705702 2026-02-17T11:50:16.707 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ats: seq=154618822682 2026-02-17T11:50:16.707 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822682 2026-02-17T11:50:16.707 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-55834574883 3-274877906961 4-98784247840 5-395136991234 6-133143986204 7-154618822682' 2026-02-17T11:50:16.707 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.707 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:50:16.707 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659161 2026-02-17T11:50:16.707 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659161 2026-02-17T11:50:16.707 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-55834574883 3-274877906961 4-98784247840 5-395136991234 6-133143986204 7-154618822682 8-176093659161' 2026-02-17T11:50:16.708 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:16.708 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:50:16.708 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=382252089346 2026-02-17T11:50:16.708 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 382252089346 2026-02-17T11:50:16.708 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836519 1-38654705701 2-55834574883 3-274877906961 4-98784247840 5-395136991234 6-133143986204 7-154618822682 8-176093659161 9-382252089346' 2026-02-17T11:50:16.708 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:16.708 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836519 2026-02-17T11:50:16.708 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:16.708 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:50:16.708 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836519 2026-02-17T11:50:16.708 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:16.708 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836519 2026-02-17T11:50:16.708 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836519' 2026-02-17T11:50:16.709 INFO:tasks.workunit.client.0.trial128.stderr:232634: waiting osd.0 seq 21474836519 2026-02-17T11:50:16.709 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:50:16.709 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836520 -lt 21474836519 2026-02-17T11:50:16.709 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:16.709 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705701 2026-02-17T11:50:16.709 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:16.709 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:50:16.709 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705701 2026-02-17T11:50:16.709 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:16.709 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705701 2026-02-17T11:50:16.709 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705701' 2026-02-17T11:50:16.709 INFO:tasks.workunit.client.0.trial128.stderr:232634: waiting osd.1 seq 38654705701 2026-02-17T11:50:17.321 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: cceph osd last-stat-seq 1 2026-02-17T11:50:17.321 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705702 -lt 38654705702 2026-02-17T11:50:17.321 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:17.321 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574884 2026-02-17T11:50:17.322 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:17.322 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:50:17.322 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574884 2026-02-17T11:50:17.322 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:17.322 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574884 2026-02-17T11:50:17.322 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574884' 2026-02-17T11:50:17.322 INFO:tasks.workunit.client.0.trial128.stderr:232632: waiting osd.2 seq 55834574884 2026-02-17T11:50:17.322 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:50:17.322 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574884 -lt 55834574884 2026-02-17T11:50:17.322 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:17.322 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-274877906962 2026-02-17T11:50:17.323 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:17.323 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:50:17.323 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-274877906962 2026-02-17T11:50:17.323 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:17.323 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=274877906962 2026-02-17T11:50:17.323 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 274877906962' 2026-02-17T11:50:17.323 INFO:tasks.workunit.client.0.trial128.stderr:232632: waiting osd.3 seq 274877906962 2026-02-17T11:50:17.323 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:50:17.323 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 274877906962 -lt 274877906962 2026-02-17T11:50:17.323 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:17.323 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247841 2026-02-17T11:50:17.323 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:17.324 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:50:17.324 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247841 2026-02-17T11:50:17.324 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:17.324 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247841 2026-02-17T11:50:17.324 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247841' 2026-02-17T11:50:17.324 INFO:tasks.workunit.client.0.trial128.stderr:232632: waiting osd.4 seq 98784247841 2026-02-17T11:50:17.324 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:50:17.324 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247841 -lt 98784247841 2026-02-17T11:50:17.324 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:17.324 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-395136991235 2026-02-17T11:50:17.349 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.ceph osd last-stat-seq 1 2026-02-17T11:50:17.349 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705702 -lt 38654705701 2026-02-17T11:50:17.349 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:17.349 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574883 2026-02-17T11:50:17.349 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:17.349 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:50:17.349 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574883 2026-02-17T11:50:17.349 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:17.349 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574883 2026-02-17T11:50:17.350 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574883' 2026-02-17T11:50:17.350 INFO:tasks.workunit.client.0.trial128.stderr:232634: waiting osd.2 seq 55834574883 2026-02-17T11:50:17.350 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:50:17.350 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574884 -lt 55834574883 2026-02-17T11:50:17.350 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:17.350 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-274877906961 2026-02-17T11:50:17.350 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:17.350 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:50:17.350 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-274877906961 2026-02-17T11:50:17.350 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:17.350 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=274877906961 2026-02-17T11:50:17.350 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 274877906961' 2026-02-17T11:50:17.351 INFO:tasks.workunit.client.0.trial128.stderr:232634: waiting osd.3 seq 274877906961 2026-02-17T11:50:17.351 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:50:17.351 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 274877906962 -lt 274877906961 2026-02-17T11:50:17.351 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:17.351 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247840 2026-02-17T11:50:17.351 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:17.351 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:50:17.351 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247840 2026-02-17T11:50:17.351 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:17.351 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247840 2026-02-17T11:50:17.351 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247840' 2026-02-17T11:50:17.351 INFO:tasks.workunit.client.0.trial128.stderr:232634: waiting osd.4 seq 98784247840 2026-02-17T11:50:17.351 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:50:17.352 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247841 -lt 98784247840 2026-02-17T11:50:17.352 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:17.352 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-395136991234 2026-02-17T11:50:18.957 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.cllient.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:18.957 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:50:18.957 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-395136991235 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=395136991235 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 395136991235' 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: waiting osd.5 seq 395136991235 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 395136991235 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 395136991235 -lt 395136991235 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986205 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986205 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986205 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986205' 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: waiting osd.6 seq 133143986205 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986206 -lt 133143986205 2026-02-17T11:50:18.958 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:18.959 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822683 2026-02-17T11:50:18.959 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:18.959 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:50:18.959 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822683 2026-02-17T11:50:18.959 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:18.959 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822683 2026-02-17T11:50:18.959 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822683' 2026-02-17T11:50:18.959 INFO:tasks.workunit.client.0.trial128.stderr:232632: waiting osd.7 seq 154618822683 2026-02-17T11:50:18.959 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:50:18.959 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822684 -lt 154618822683 2026-02-17T11:50:18.959 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:18.963 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: ecient.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:18.963 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-395136991234 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=395136991234 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 395136991234' 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: waiting osd.5 seq 395136991234 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 395136991234 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 395136991235 -lt 395136991234 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986204 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986204 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986204 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986204' 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: waiting osd.6 seq 133143986204 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986206 -lt 133143986204 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822682 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:18.964 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:50:18.965 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822682 2026-02-17T11:50:18.965 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:18.965 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822682 2026-02-17T11:50:18.965 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822682' 2026-02-17T11:50:18.965 INFO:tasks.workunit.client.0.trial128.stderr:232634: waiting osd.7 seq 154618822682 2026-02-17T11:50:18.965 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:50:18.965 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822684 -lt 154618822682 2026-02-17T11:50:18.965 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659161 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659161 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659161 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659161' 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: waiting osd.8 seq 176093659161 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659162 -lt 176093659161 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-382252089346 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-382252089346 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=382252089346 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 382252089346' 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: waiting osd.9 seq 382252089346 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 382252089347 -lt 382252089346 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:50:19.606 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:50:19.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:50:19.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:50:19.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:50:19.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:50:19.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:50:19.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:50:19.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:50:19.607 INFO:tasks.workunit.client.0.trial128.stderr:232634: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:50:19.624 INFO:tasks.workunit.client.0.trial128.stderr:232634: //homeho 8-176093659162 2026-02-17T11:50:19.624 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:19.624 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:50:19.624 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659162 2026-02-17T11:50:19.624 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:19.624 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659162 2026-02-17T11:50:19.624 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659162' 2026-02-17T11:50:19.624 INFO:tasks.workunit.client.0.trial128.stderr:232632: waiting osd.8 seq 176093659162 2026-02-17T11:50:19.624 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659162 -lt 176093659162 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-382252089347 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-382252089347 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=382252089347 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 382252089347' 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: waiting osd.9 seq 382252089347 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 382252089347 -lt 382252089347 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:50:19.625 INFO:tasks.workunit.client.0.trial128.stderr:232632: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:50:19.788 INFO:tasks.workunit.client.0.trial128.stderr:232632: //hom/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:50:19.788 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:50:19.788 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:50:19.788 INFO:tasks.workunit.client.0.trial128.stderr:232634: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:50:19.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:50:19.813 INFO:tasks.workunit.client.0.trial128.stderr:e/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:50:19.813 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:50:19.813 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:50:19.813 INFO:tasks.workunit.client.0.trial128.stderr:232632: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:50:19.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:50:19.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:50:19.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 232631 2026-02-17T11:50:19.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T11:50:19.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T11:50:19.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T11:50:19.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: corrupt_and_repair_two: return_code=0 2026-02-17T11:50:19.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:198: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T11:50:19.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-17T11:50:19.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:50:19.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:50:19.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:50:19.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:50:19.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: local pg=2.0 2026-02-17T11:50:19.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:204: corrupt_and_repair_two: repair 2.0 2026-02-17T11:50:19.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T11:50:19.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T11:50:19.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:50:19.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:50:19.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:50:19.976 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:50:20.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:49:54.371607+0000 2026-02-17T11:50:20.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T11:50:20.272 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T11:50:20.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T11:49:54.371607+0000 2026-02-17T11:50:20.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:50:20.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:49:54.371607+0000 2026-02-17T11:50:20.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:50:20.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:50:20.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:50:20.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:50:20.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:50:20.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:50:20.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:50:20.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:50:20.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:49:54.371607+0000 '>' 2026-02-17T11:49:54.371607+0000 2026-02-17T11:50:20.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:50:21.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:50:21.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:50:21.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:50:21.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:50:21.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:50:21.443 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:50:21.443 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:50:21.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:49:54.371607+0000 '>' 2026-02-17T11:49:54.371607+0000 2026-02-17T11:50:21.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:50:22.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:50:22.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:50:22.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:50:22.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:50:22.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:50:22.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:50:22.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:50:22.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:49:54.371607+0000 '>' 2026-02-17T11:49:54.371607+0000 2026-02-17T11:50:22.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:50:23.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:50:23.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:50:23.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:50:23.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:50:23.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:50:23.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:50:23.759 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:50:23.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:49:54.371607+0000 '>' 2026-02-17T11:49:54.371607+0000 2026-02-17T11:50:23.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:50:24.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:50:24.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:50:24.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:50:24.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:50:24.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:50:24.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:50:24.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:50:25.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:49:54.371607+0000 '>' 2026-02-17T11:49:54.371607+0000 2026-02-17T11:50:25.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:50:26.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:50:26.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:50:26.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:50:26.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:50:26.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:50:26.074 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:50:26.074 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:50:26.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:50:20.429951+0000 '>' 2026-02-17T11:49:54.371607+0000 2026-02-17T11:50:26.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:50:26.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: corrupt_and_repair_two: pids= 2026-02-17T11:50:26.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T11:50:26.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:50:26.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:50:26.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 236210"' 2026-02-17T11:50:26.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 236210' 2026-02-17T11:50:26.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:50:26.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:50:26.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:50:26.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 236212"' 2026-02-17T11:50:26.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T11:50:26.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 236212' 2026-02-17T11:50:26.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:211: corrupt_and_repair_two: wait_background pids 2026-02-17T11:50:26.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 236210 236212' 2026-02-17T11:50:26.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T11:50:26.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:50:26.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 236210 2026-02-17T11:50:26.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/236213: /' 2026-02-17T11:50:26.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:50:26.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/236216: /' 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T11:50:28.389 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:50:28.390 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:50:28.390 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING list-attrs 2026-02-17T11:50:28.390 INFO:tasks.workunit.client.0.trial128.stderr:236213: _ 2026-02-17T11:50:28.390 INFO:tasks.workunit.client.0.trial128.stderr:236213: hinfo_key 2026-02-17T11:50:28.390 INFO:tasks.workunit.client.0.trial128.stderr:236213: snapset 2026-02-17T11:50:28.390 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T11:50:28.390 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:50:28.390 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:50:28.390 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T11:50:28.390 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:50:28.390 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:50:28.390 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:50:28.390 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T11:50:28.392 INFO:tasks.workunit.client.0.trial128.stderr:236213: start osd.5 2026-02-17T11:50:28.437 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_rec236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:50:28.437 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=9 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=9 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.9 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 9 SOMETHING list-attrs 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=9 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/9 SOMETHING list-attrs 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: _ 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: hinfo_key 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: snapset 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 9 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:50:28.438 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:50:28.439 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=9 2026-02-17T11:50:28.439 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:50:28.439 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/9 2026-02-17T11:50:28.439 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:50:28.439 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:50:28.440 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:50:28.440 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:50:28.441 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/9' 2026-02-17T11:50:28.441 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/9/journal' 2026-02-17T11:50:28.441 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:50:28.441 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:50:28.441 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:50:28.441 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:50:28.441 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:50:28.441 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:50:28.441 INFO:tasks.workunit.client.0.trial128.stderr:236216: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:50:28.441 INFO:tasks.workunit.client.0.trial128.stderr:236216: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:50:28.441 INFO:tasks.workunit.client.0.trial128.stderr:236216: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:50:28.441 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:50:28.442 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:50:28.442 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:50:28.442 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:50:28.442 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:50:28.442 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:50:28.442 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:50:28.442 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:50:28.442 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:50:28.442 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:50:28.442 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:50:28.442 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/9 2026-02-17T11:50:28.442 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.9 2026-02-17T11:50:28.442 INFO:tasks.workunit.client.0.trial128.stderr:236216: start osd.9 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 9 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/9 --osd-journal=td/osd-scrub-repair/9/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: 2026-02-17T11:50:28.400+0000 7f795c1028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: 2026-02-17T11:50:28.400+0000 7f795c1028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: 2026-02-17T11:50:28.400+0000 7f795c1028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: 0 2026-02-17T11:50:32.768 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: 1 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: 2026-02-17T11:50:29.728+0000 7f795c1028c0 -1 Falling back to public interface 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: 2 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: 3 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:50:32.769 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:33.060 INFO:tasks.workunit.client.0.trial128.stderr:236213: 2026-02-17T11:50:32.756+0000 7f7overy_ops 2026-02-17T11:50:33.061 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/9/whoami 2026-02-17T11:50:33.061 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 9 = 9 ']' 2026-02-17T11:50:33.061 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:50:33.061 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:50:33.061 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:50:33.061 INFO:tasks.workunit.client.0.trial128.stderr:236216: 2026-02-17T11:50:28.444+0000 7ff2fecb28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:33.061 INFO:tasks.workunit.client.0.trial128.stderr:236216: 2026-02-17T11:50:28.444+0000 7ff2fecb28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:33.061 INFO:tasks.workunit.client.0.trial128.stderr:236216: 2026-02-17T11:50:28.444+0000 7ff2fecb28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:33.062 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 9 2026-02-17T11:50:33.062 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:50:33.062 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=9 2026-02-17T11:50:33.062 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:50:33.062 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:50:33.062 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:33.062 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:50:33.062 INFO:tasks.workunit.client.0.trial128.stderr:236216: 0 2026-02-17T11:50:33.062 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:33.062 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:50:33.062 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:33.063 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:33.063 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:33.063 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:50:33.063 INFO:tasks.workunit.client.0.trial128.stderr:236216: 1 2026-02-17T11:50:33.063 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:33.063 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:50:33.063 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:33.063 INFO:tasks.workunit.client.0.trial128.stderr:236216: 2026-02-17T11:50:30.000+0000 7ff2fecb28c0 -1 Falling back to public interface 2026-02-17T11:50:33.063 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:33.063 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:33.063 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:50:33.063 INFO:tasks.workunit.client.0.trial128.stderr:236216: 2 2026-02-17T11:50:33.064 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:33.064 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:50:33.064 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:33.064 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:33.064 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:33.064 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:50:33.064 INFO:tasks.workunit.client.0.trial128.stderr:236216: 3 2026-02-17T11:50:33.064 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:33.064 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:50:33.064 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:33.661 INFO:tasks.workunit.client.0.trial128.stderr:236216: 2026-02-17T11:50:33.048+0000 7ff95c1028c0 -1 osd.5 93 log_to_monitors true 2026-02-17T11:50:33.661 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:33.661 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:33.662 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:50:33.662 INFO:tasks.workunit.client.0.trial128.stderr:236213: 4 2026-02-17T11:50:33.662 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:33.662 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:50:33.662 INFO:tasks.workunit.client.0.trial128.stderr:236213: osd.5 up in weight 1 up_from 98 up_thru 98 down_at 94 last_clean_interval [92,93) [v2:127.0.0.1:6842/3984943224,v1:127.0.0.1:6843/3984943224] [v2:127.0.0.1:6844/3984943224,v1:127.0.0.1:6845/3984943224] exists,up 80987ebb-2096-4a73-b7c5-64048030732e 2026-02-17T11:50:33.662 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:50:33.663 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:50:33.663 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:50:33.663 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:50:33.663 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:50:33.663 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:50:33.663 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:50:33.663 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:50:33.663 INFO:tasks.workunit.client.0.trial128.stderr:236213: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:50:33.663 INFO:tasks.workunit.client.0.trial128.stderr:236213: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:50:33.663 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:50:33.664 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:50:33.664 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:50:33.664 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:50:33.664 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:50:33.664 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:50:33.664 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:50:33.664 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:50:33.664 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:50:33.664 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:50:33.664 INFO:tasks.workunit.client.0.trial128.stderr:236213: 1 2026-02-17T11:50:33.664 INFO:tasks.workunit.client.0.trial128.stderr:236213: 2 2026-02-17T11:50:33.664 INFO:tasks.workunit.client.0.trial128.stderr:236213: 3 2026-02-17T11:50:33.665 INFO:tasks.workunit.client.0.trial128.stderr:236213: 4 2026-02-17T11:50:33.665 INFO:tasks.workunit.client.0.trial128.stderr:236213: 5 2026-02-17T11:50:33.665 INFO:tasks.workunit.client.0.trial128.stderr:236213: 6 2026-02-17T11:50:33.665 INFO:tasks.workunit.client.0.trial128.stderr:236213: 7 2026-02-17T11:50:33.665 INFO:tasks.workunit.client.0.trial128.stderr:236213: 8 2026-02-17T11:50:33.665 INFO:tasks.workunit.client.0.trial128.stderr:236213: 9' 2026-02-17T11:50:33.665 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:50:33.665 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:33.665 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:50:33.665 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836524 2026-02-17T11:50:33.665 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836524 2026-02-17T11:50:33.666 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524' 2026-02-17T11:50:33.711 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $id2fecb28c0 -1 osd.9 93 log_to_monitors true 2026-02-17T11:50:33.711 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:33.711 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:33.711 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:50:33.711 INFO:tasks.workunit.client.0.trial128.stderr:236216: 4 2026-02-17T11:50:33.711 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:33.711 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.9 up' 2026-02-17T11:50:33.711 INFO:tasks.workunit.client.0.trial128.stderr:236216: osd.9 up in weight 1 up_from 101 up_thru 0 down_at 94 last_clean_interval [89,93) [v2:127.0.0.1:6874/559502671,v1:127.0.0.1:6875/559502671] [v2:127.0.0.1:6876/559502671,v1:127.0.0.1:6877/559502671] exists,up 6ca155d1-18e1-4577-a926-edebebb3c55d 2026-02-17T11:50:33.711 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:50:33.711 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:50:33.711 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:50:33.711 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:50:33.711 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:50:33.711 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:50:33.711 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: 1 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: 2 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: 3 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: 4 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: 5 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: 6 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: 7 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: 8 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: 9' 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836525 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836525 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525' 2026-02-17T11:50:33.712 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.161 INFO:tasks.workunit.client.0.trial128.stderr:23s 2026-02-17T11:50:34.162 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:50:34.162 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705706 2026-02-17T11:50:34.162 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705706 2026-02-17T11:50:34.162 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706' 2026-02-17T11:50:34.162 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.162 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:50:34.162 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574889 2026-02-17T11:50:34.162 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574889 2026-02-17T11:50:34.163 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-55834574889' 2026-02-17T11:50:34.163 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.163 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:50:34.163 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=274877906966 2026-02-17T11:50:34.163 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 274877906966 2026-02-17T11:50:34.163 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-55834574889 3-274877906966' 2026-02-17T11:50:34.163 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.163 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:50:34.163 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247845 2026-02-17T11:50:34.163 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247845 2026-02-17T11:50:34.163 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-55834574889 3-274877906966 4-98784247845' 2026-02-17T11:50:34.163 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.164 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:50:34.164 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=420906795010 2026-02-17T11:50:34.164 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 420906795010 2026-02-17T11:50:34.164 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-55834574889 3-274877906966 4-98784247845 5-420906795010' 2026-02-17T11:50:34.164 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.164 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:50:34.164 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986210 2026-02-17T11:50:34.164 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986210 2026-02-17T11:50:34.164 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-55834574889 3-274877906966 4-98784247845 5-420906795010 6-133143986210' 2026-02-17T11:50:34.164 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.164 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:50:34.233 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: 6216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:50:34.234 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705707 2026-02-17T11:50:34.234 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705707 2026-02-17T11:50:34.234 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707' 2026-02-17T11:50:34.234 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.234 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:50:34.234 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574890 2026-02-17T11:50:34.234 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574890 2026-02-17T11:50:34.234 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707 2-55834574890' 2026-02-17T11:50:34.234 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.235 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:50:34.235 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=274877906967 2026-02-17T11:50:34.235 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 274877906967 2026-02-17T11:50:34.235 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707 2-55834574890 3-274877906967' 2026-02-17T11:50:34.235 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.235 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:50:34.235 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247846 2026-02-17T11:50:34.235 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247846 2026-02-17T11:50:34.235 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707 2-55834574890 3-274877906967 4-98784247846' 2026-02-17T11:50:34.235 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.235 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:50:34.235 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=420906795011 2026-02-17T11:50:34.235 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 420906795011 2026-02-17T11:50:34.236 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707 2-55834574890 3-274877906967 4-98784247846 5-420906795011' 2026-02-17T11:50:34.236 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.236 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:50:34.236 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986211 2026-02-17T11:50:34.236 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986211 2026-02-17T11:50:34.236 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707 2-55834574890 3-274877906967 4-98784247846 5-420906795011 6-133143986211' 2026-02-17T11:50:34.236 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.236 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flusflush_pg_stats: seq=154618822688 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822688 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-55834574889 3-274877906966 4-98784247845 5-420906795010 6-133143986210 7-154618822688' 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659166 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659166 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-55834574889 3-274877906966 4-98784247845 5-420906795010 6-133143986210 7-154618822688 8-176093659166' 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=433791696898 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 433791696898 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-38654705706 2-55834574889 3-274877906966 4-98784247845 5-420906795010 6-133143986210 7-154618822688 8-176093659166 9-433791696898' 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836524 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836524 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836524 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836524' 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: waiting osd.0 seq 21474836524 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836525 -lt 21474836524 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:34.476 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705706 2026-02-17T11:50:34.477 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:34.477 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:50:34.477 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705706 2026-02-17T11:50:34.477 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:34.477 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705706 2026-02-17T11:50:34.477 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705706' 2026-02-17T11:50:34.477 INFO:tasks.workunit.client.0.trial128.stderr:236213: waiting osd.1 seq 38654705706 2026-02-17T11:50:34.536 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_ph_pg_stats: seq=154618822689 2026-02-17T11:50:34.536 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822689 2026-02-17T11:50:34.536 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707 2-55834574890 3-274877906967 4-98784247846 5-420906795011 6-133143986211 7-154618822689' 2026-02-17T11:50:34.536 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.536 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:50:34.536 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659167 2026-02-17T11:50:34.536 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659167 2026-02-17T11:50:34.536 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707 2-55834574890 3-274877906967 4-98784247846 5-420906795011 6-133143986211 7-154618822689 8-176093659167' 2026-02-17T11:50:34.536 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:34.537 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:50:34.537 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=433791696899 2026-02-17T11:50:34.537 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 433791696899 2026-02-17T11:50:34.537 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705707 2-55834574890 3-274877906967 4-98784247846 5-420906795011 6-133143986211 7-154618822689 8-176093659167 9-433791696899' 2026-02-17T11:50:34.537 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:34.537 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836525 2026-02-17T11:50:34.537 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:34.537 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:50:34.537 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836525 2026-02-17T11:50:34.537 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:34.537 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836525 2026-02-17T11:50:34.537 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836525' 2026-02-17T11:50:34.537 INFO:tasks.workunit.client.0.trial128.stderr:236216: waiting osd.0 seq 21474836525 2026-02-17T11:50:34.538 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:50:34.538 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836525 -lt 21474836525 2026-02-17T11:50:34.538 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:34.538 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705707 2026-02-17T11:50:34.538 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:34.538 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:50:34.538 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705707 2026-02-17T11:50:34.538 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:34.538 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705707 2026-02-17T11:50:34.538 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705707' 2026-02-17T11:50:34.538 INFO:tasks.workunit.client.0.trial128.stderr:236216: waiting osd.1 seq 38654705707 2026-02-17T11:50:35.123 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stg_stats: ceph osd last-stat-seq 1 2026-02-17T11:50:35.123 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705707 -lt 38654705706 2026-02-17T11:50:35.123 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:35.123 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574889 2026-02-17T11:50:35.123 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:35.124 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:50:35.124 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574889 2026-02-17T11:50:35.124 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:35.124 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574889 2026-02-17T11:50:35.124 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574889' 2026-02-17T11:50:35.124 INFO:tasks.workunit.client.0.trial128.stderr:236213: waiting osd.2 seq 55834574889 2026-02-17T11:50:35.124 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:50:35.124 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574890 -lt 55834574889 2026-02-17T11:50:35.124 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:35.124 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-274877906966 2026-02-17T11:50:35.125 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:35.125 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:50:35.125 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-274877906966 2026-02-17T11:50:35.125 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:35.125 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=274877906966 2026-02-17T11:50:35.125 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 274877906966' 2026-02-17T11:50:35.125 INFO:tasks.workunit.client.0.trial128.stderr:236213: waiting osd.3 seq 274877906966 2026-02-17T11:50:35.125 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:50:35.125 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 274877906967 -lt 274877906966 2026-02-17T11:50:35.125 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:35.125 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247845 2026-02-17T11:50:35.125 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:35.125 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:50:35.126 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247845 2026-02-17T11:50:35.126 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:35.126 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247845 2026-02-17T11:50:35.126 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247845' 2026-02-17T11:50:35.126 INFO:tasks.workunit.client.0.trial128.stderr:236213: waiting osd.4 seq 98784247845 2026-02-17T11:50:35.126 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:50:35.126 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247846 -lt 98784247845 2026-02-17T11:50:35.126 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:35.126 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-420906795010 2026-02-17T11:50:35.166 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephteats: ceph osd last-stat-seq 1 2026-02-17T11:50:35.166 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705707 -lt 38654705707 2026-02-17T11:50:35.167 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:35.167 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574890 2026-02-17T11:50:35.167 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:35.167 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:50:35.167 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574890 2026-02-17T11:50:35.167 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:35.167 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574890 2026-02-17T11:50:35.167 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574890' 2026-02-17T11:50:35.167 INFO:tasks.workunit.client.0.trial128.stderr:236216: waiting osd.2 seq 55834574890 2026-02-17T11:50:35.167 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:50:35.167 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574890 -lt 55834574890 2026-02-17T11:50:35.168 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:35.168 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-274877906967 2026-02-17T11:50:35.168 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:35.168 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:50:35.168 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-274877906967 2026-02-17T11:50:35.168 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:35.168 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=274877906967 2026-02-17T11:50:35.168 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 274877906967' 2026-02-17T11:50:35.168 INFO:tasks.workunit.client.0.trial128.stderr:236216: waiting osd.3 seq 274877906967 2026-02-17T11:50:35.168 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:50:35.168 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 274877906967 -lt 274877906967 2026-02-17T11:50:35.168 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:35.168 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247846 2026-02-17T11:50:35.169 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:35.169 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:50:35.169 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247846 2026-02-17T11:50:35.169 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:35.169 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247846 2026-02-17T11:50:35.169 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247846' 2026-02-17T11:50:35.169 INFO:tasks.workunit.client.0.trial128.stderr:236216: waiting osd.4 seq 98784247846 2026-02-17T11:50:35.169 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:50:35.169 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247846 -lt 98784247846 2026-02-17T11:50:35.169 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:35.169 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-420906795011 2026-02-17T11:50:35.603 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/cst/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:35.603 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:50:35.603 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-420906795010 2026-02-17T11:50:35.603 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:35.603 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=420906795010 2026-02-17T11:50:35.603 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 420906795010' 2026-02-17T11:50:35.603 INFO:tasks.workunit.client.0.trial128.stderr:236213: waiting osd.5 seq 420906795010 2026-02-17T11:50:35.603 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:50:35.604 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 420906795011 -lt 420906795010 2026-02-17T11:50:35.604 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:35.604 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986210 2026-02-17T11:50:35.604 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:35.604 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:50:35.604 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986210 2026-02-17T11:50:35.604 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:35.604 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986210 2026-02-17T11:50:35.604 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986210' 2026-02-17T11:50:35.604 INFO:tasks.workunit.client.0.trial128.stderr:236213: waiting osd.6 seq 133143986210 2026-02-17T11:50:35.604 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:50:35.604 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986211 -lt 133143986210 2026-02-17T11:50:35.605 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:35.605 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822688 2026-02-17T11:50:35.605 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:35.605 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:50:35.605 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822688 2026-02-17T11:50:35.605 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:35.605 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822688 2026-02-17T11:50:35.605 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822688' 2026-02-17T11:50:35.605 INFO:tasks.workunit.client.0.trial128.stderr:236213: waiting osd.7 seq 154618822688 2026-02-17T11:50:35.605 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:50:35.605 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822688 -lt 154618822688 2026-02-17T11:50:35.605 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:35.605 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659166 2026-02-17T11:50:35.606 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:35.606 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:50:35.606 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659166 2026-02-17T11:50:36.795 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -lone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:36.795 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:50:36.796 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-420906795011 2026-02-17T11:50:36.796 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:36.796 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=420906795011 2026-02-17T11:50:36.796 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 420906795011' 2026-02-17T11:50:36.796 INFO:tasks.workunit.client.0.trial128.stderr:236216: waiting osd.5 seq 420906795011 2026-02-17T11:50:36.796 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:50:36.796 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 420906795011 -lt 420906795011 2026-02-17T11:50:36.796 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:36.796 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986211 2026-02-17T11:50:36.796 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:36.796 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:50:36.797 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986211 2026-02-17T11:50:36.797 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:36.797 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986211 2026-02-17T11:50:36.797 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986211' 2026-02-17T11:50:36.797 INFO:tasks.workunit.client.0.trial128.stderr:236216: waiting osd.6 seq 133143986211 2026-02-17T11:50:36.797 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:50:36.797 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986211 -lt 133143986211 2026-02-17T11:50:36.797 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:36.797 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822689 2026-02-17T11:50:36.797 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:36.797 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:50:36.797 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822689 2026-02-17T11:50:36.797 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:36.798 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822689 2026-02-17T11:50:36.798 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822689' 2026-02-17T11:50:36.798 INFO:tasks.workunit.client.0.trial128.stderr:236216: waiting osd.7 seq 154618822689 2026-02-17T11:50:36.798 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:50:36.798 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822688 -lt 154618822689 2026-02-17T11:50:36.798 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:50:36.798 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:50:36.798 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:50:36.798 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822689 -lt 154618822689 2026-02-17T11:50:36.798 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:37.432 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: fld - -f 2 2026-02-17T11:50:37.432 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659166 2026-02-17T11:50:37.432 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659166' 2026-02-17T11:50:37.432 INFO:tasks.workunit.client.0.trial128.stderr:236213: waiting osd.8 seq 176093659166 2026-02-17T11:50:37.432 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:50:37.432 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659165 -lt 176093659166 2026-02-17T11:50:37.432 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:50:37.432 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:50:37.433 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:50:37.433 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659168 -lt 176093659166 2026-02-17T11:50:37.433 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:37.433 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-433791696898 2026-02-17T11:50:37.433 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:37.433 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:50:37.433 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-433791696898 2026-02-17T11:50:37.433 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:37.433 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=433791696898 2026-02-17T11:50:37.433 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 433791696898' 2026-02-17T11:50:37.433 INFO:tasks.workunit.client.0.trial128.stderr:236213: waiting osd.9 seq 433791696898 2026-02-17T11:50:37.433 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:50:37.433 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 433791696899 -lt 433791696898 2026-02-17T11:50:37.434 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:50:37.434 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:50:37.434 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:50:37.434 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:50:37.434 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:50:37.434 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:50:37.434 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:50:37.434 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:50:37.434 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:50:37.434 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:50:37.434 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:50:37.435 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:50:37.435 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:50:37.445 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format jsonush_pg_stats: echo 8-176093659167 2026-02-17T11:50:37.445 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:37.445 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:50:37.446 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659167 2026-02-17T11:50:37.446 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:37.446 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659167 2026-02-17T11:50:37.446 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659167' 2026-02-17T11:50:37.446 INFO:tasks.workunit.client.0.trial128.stderr:236216: waiting osd.8 seq 176093659167 2026-02-17T11:50:37.446 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:50:37.446 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659168 -lt 176093659167 2026-02-17T11:50:37.446 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:37.446 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-433791696899 2026-02-17T11:50:37.446 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:37.446 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:50:37.447 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-433791696899 2026-02-17T11:50:37.447 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:37.447 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=433791696899 2026-02-17T11:50:37.447 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 433791696899' 2026-02-17T11:50:37.447 INFO:tasks.workunit.client.0.trial128.stderr:236216: waiting osd.9 seq 433791696899 2026-02-17T11:50:37.447 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:50:37.447 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 433791696899 -lt 433791696899 2026-02-17T11:50:37.447 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:50:37.447 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:50:37.447 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:50:37.447 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:50:37.447 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:50:37.447 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:50:37.448 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:50:37.448 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:50:37.448 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:50:37.448 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:50:37.448 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:50:37.448 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:50:37.448 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:50:37.621 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json sta status 2026-02-17T11:50:37.621 INFO:tasks.workunit.client.0.trial128.stderr:236213: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:50:37.621 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:50:37.622 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:50:37.622 INFO:tasks.workunit.client.0.trial128.stderr:236213: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:50:37.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:50:37.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:50:37.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 236212 2026-02-17T11:50:37.627 INFO:tasks.workunit.client.0.trial128.stderr:tus 2026-02-17T11:50:37.628 INFO:tasks.workunit.client.0.trial128.stderr:236216: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:50:37.628 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:50:37.628 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:50:37.628 INFO:tasks.workunit.client.0.trial128.stderr:236216: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:50:37.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:50:37.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T11:50:37.629 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T11:50:37.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T11:50:37.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:212: corrupt_and_repair_two: return_code=0 2026-02-17T11:50:37.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:213: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T11:50:37.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:50:37.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:50:37.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:263: corrupt_and_repair_erasure_coded: corrupt_and_repair_two td/osd-scrub-repair ecpool 3 5 2026-02-17T11:50:37.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:185: corrupt_and_repair_two: local dir=td/osd-scrub-repair 2026-02-17T11:50:37.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:186: corrupt_and_repair_two: local poolname=ecpool 2026-02-17T11:50:37.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:187: corrupt_and_repair_two: local first=3 2026-02-17T11:50:37.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:188: corrupt_and_repair_two: local second=5 2026-02-17T11:50:37.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:193: corrupt_and_repair_two: pids= 2026-02-17T11:50:37.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:194: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:50:37.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:50:37.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:50:37.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 239227"' 2026-02-17T11:50:37.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 239227' 2026-02-17T11:50:37.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:195: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:50:37.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:50:37.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:50:37.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 239228"' 2026-02-17T11:50:37.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 239228' 2026-02-17T11:50:37.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:196: corrupt_and_repair_two: wait_background pids 2026-02-17T11:50:37.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 239227 239228' 2026-02-17T11:50:37.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T11:50:37.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:50:37.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 239227 2026-02-17T11:50:37.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:50:37.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/239230: /' 2026-02-17T11:50:37.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:50:37.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/239232: /' 2026-02-17T11:50:40.129 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:50:40.129 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:50:40.129 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T11:50:40.129 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: remove 0#2:eb822e21:::SOMETHING:head# 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:50:40.130 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:50:40.133 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:50:40.134 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:50:40.134 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:50:40.134 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:50:40.134 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:50:40.134 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:50:40.134 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:50:40.134 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:50:40.134 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:50:40.134 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T11:50:40.134 INFO:tasks.workunit.client.0.trial128.stderr:239230: start osd.3 2026-02-17T11:50:40.134 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:50:40.194 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:50:40.194 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:50:40.194 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T11:50:40.194 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:50:40.194 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:50:40.194 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:50:40.194 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:50:40.194 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T11:50:40.194 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:50:40.194 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T11:50:40.194 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:50:40.194 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:50:40.194 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:50:40.194 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:50:40.194 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:50:40.194 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:50:40.195 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING remove 2026-02-17T11:50:40.195 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:50:40.195 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:50:40.195 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T11:50:40.195 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:50:40.195 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:50:40.195 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING remove 2026-02-17T11:50:40.195 INFO:tasks.workunit.client.0.trial128.stderr:239232: remove 1#2:eb822e21:::SOMETHING:head# 2026-02-17T11:50:40.195 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T11:50:40.195 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:50:40.195 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:50:40.195 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T11:50:40.195 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:50:40.195 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:50:40.195 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:50:40.195 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:50:40.197 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:50:40.197 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:50:40.197 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T11:50:40.197 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T11:50:40.197 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:50:40.197 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:50:40.197 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:50:40.197 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:50:40.197 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: start osd.5 2026-02-17T11:50:40.198 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:50:44.453 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T11:50:44.453 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T11:50:44.453 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:50:44.453 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:50:44.453 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:50:44.453 INFO:tasks.workunit.client.0.trial128.stderr:239230: 2026-02-17T11:50:40.144+0000 7fd87a24d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:44.453 INFO:tasks.workunit.client.0.trial128.stderr:239230: 2026-02-17T11:50:40.144+0000 7fd87a24d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:44.453 INFO:tasks.workunit.client.0.trial128.stderr:239230: 2026-02-17T11:50:40.148+0000 7fd87a24d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:44.453 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T11:50:44.454 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:50:44.454 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:50:44.454 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:50:44.454 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:50:44.454 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:44.454 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:50:44.454 INFO:tasks.workunit.client.0.trial128.stderr:239230: 0 2026-02-17T11:50:44.454 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:44.454 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:50:44.454 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:44.454 INFO:tasks.workunit.client.0.trial128.stderr:239230: 2026-02-17T11:50:41.412+0000 7fd87a24d8c0 -1 Falling back to public interface 2026-02-17T11:50:44.454 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:44.455 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:44.455 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:50:44.455 INFO:tasks.workunit.client.0.trial128.stderr:239230: 1 2026-02-17T11:50:44.455 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:44.455 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:50:44.455 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:44.455 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:44.455 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:44.455 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:50:44.455 INFO:tasks.workunit.client.0.trial128.stderr:239230: 2 2026-02-17T11:50:44.455 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:44.455 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:50:44.456 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:44.456 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:44.456 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:44.456 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:50:44.456 INFO:tasks.workunit.client.0.trial128.stderr:239230: 3 2026-02-17T11:50:44.456 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:44.456 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:50:44.456 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239230: 2026-02-17T11:50:44.440+0000 7fd87a24d8c0 239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: 2026-02-17T11:50:40.200+0000 7f43802f98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: 2026-02-17T11:50:40.200+0000 7f43802f98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: 2026-02-17T11:50:40.204+0000 7f43802f98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: 0 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: 1 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: 2026-02-17T11:50:41.744+0000 7f43802f98c0 -1 Falling back to public interface 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: 2 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:44.847 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:44.848 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:50:44.848 INFO:tasks.workunit.client.0.trial128.stderr:239232: 3 2026-02-17T11:50:44.848 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:44.848 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:50:44.848 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:50:45.393 INFO:tasks.workunit.client.0.trial128.stderr:239232: 2026-02-17T11:50:44.836+0000 7f43802f98c0 -1 osd.3 102 log_to_monitors true 2026-02-17T11:50:45.393 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:45.393 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:45.394 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:50:45.394 INFO:tasks.workunit.client.0.trial128.stderr:239230: 4 2026-02-17T11:50:45.394 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:45.394 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:50:45.394 INFO:tasks.workunit.client.0.trial128.stderr:239230: osd.3 up in weight 1 up_from 107 up_thru 107 down_at 103 last_clean_interval [64,102) [v2:127.0.0.1:6826/2303897003,v1:127.0.0.1:6827/2303897003] [v2:127.0.0.1:6828/2303897003,v1:127.0.0.1:6829/2303897003] exists,up 063dd7f2-c316-40f3-bf48-6933c998a894 2026-02-17T11:50:45.394 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:50:45.394 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:50:45.394 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:50:45.394 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:50:45.394 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:50:45.394 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:50:45.395 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:50:45.395 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:50:45.395 INFO:tasks.workunit.client.0.trial128.stderr:239230: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:50:45.395 INFO:tasks.workunit.client.0.trial128.stderr:239230: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:50:45.395 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:50:45.395 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:50:45.395 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:50:45.395 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:50:45.395 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:50:45.395 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:50:45.395 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:50:45.395 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:50:45.396 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:50:45.396 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:50:45.396 INFO:tasks.workunit.client.0.trial128.stderr:239230: 1 2026-02-17T11:50:45.396 INFO:tasks.workunit.client.0.trial128.stderr:239230: 2 2026-02-17T11:50:45.396 INFO:tasks.workunit.client.0.trial128.stderr:239230: 3 2026-02-17T11:50:45.396 INFO:tasks.workunit.client.0.trial128.stderr:239230: 4 2026-02-17T11:50:45.396 INFO:tasks.workunit.client.0.trial128.stderr:239230: 5 2026-02-17T11:50:45.396 INFO:tasks.workunit.client.0.trial128.stderr:239230: 6 2026-02-17T11:50:45.396 INFO:tasks.workunit.client.0.trial128.stderr:239230: 7 2026-02-17T11:50:45.396 INFO:tasks.workunit.client.0.trial128.stderr:239230: 8 2026-02-17T11:50:45.396 INFO:tasks.workunit.client.0.trial128.stderr:239230: 9' 2026-02-17T11:50:45.396 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:50:45.397 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:45.397 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:50:45.397 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836528 2026-02-17T11:50:45.397 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836528 2026-02-17T11:50:45.397 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528' 2026-02-17T11:50:45.397 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:45.460 INFO:tasks.workunit.client.0.trial128.stderr:239-1 osd.5 102 log_to_monitors true 2026-02-17T11:50:45.460 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:50:45.460 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:50:45.460 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:50:45.460 INFO:tasks.workunit.client.0.trial128.stderr:239232: 4 2026-02-17T11:50:45.461 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:50:45.461 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:50:45.461 INFO:tasks.workunit.client.0.trial128.stderr:239232: osd.5 up in weight 1 up_from 110 up_thru 110 down_at 103 last_clean_interval [98,102) [v2:127.0.0.1:6842/1451824626,v1:127.0.0.1:6843/1451824626] [v2:127.0.0.1:6844/1451824626,v1:127.0.0.1:6845/1451824626] exists,up 80987ebb-2096-4a73-b7c5-64048030732e 2026-02-17T11:50:45.461 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:50:45.461 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:50:45.461 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:50:45.461 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:50:45.461 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:50:45.461 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:50:45.461 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:50:45.461 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:50:45.462 INFO:tasks.workunit.client.0.trial128.stderr:239232: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:50:45.462 INFO:tasks.workunit.client.0.trial128.stderr:239232: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:50:45.462 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:50:45.462 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:50:45.462 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:50:45.462 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:50:45.462 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:50:45.462 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:50:45.462 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:50:45.462 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:50:45.462 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:50:45.462 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:50:45.462 INFO:tasks.workunit.client.0.trial128.stderr:239232: 1 2026-02-17T11:50:45.463 INFO:tasks.workunit.client.0.trial128.stderr:239232: 2 2026-02-17T11:50:45.463 INFO:tasks.workunit.client.0.trial128.stderr:239232: 3 2026-02-17T11:50:45.463 INFO:tasks.workunit.client.0.trial128.stderr:239232: 4 2026-02-17T11:50:45.463 INFO:tasks.workunit.client.0.trial128.stderr:239232: 5 2026-02-17T11:50:45.463 INFO:tasks.workunit.client.0.trial128.stderr:239232: 6 2026-02-17T11:50:45.463 INFO:tasks.workunit.client.0.trial128.stderr:239232: 7 2026-02-17T11:50:45.463 INFO:tasks.workunit.client.0.trial128.stderr:239232: 8 2026-02-17T11:50:45.463 INFO:tasks.workunit.client.0.trial128.stderr:239232: 9' 2026-02-17T11:50:45.463 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:50:45.463 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:45.463 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:50:45.463 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836529 2026-02-17T11:50:45.463 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836529 2026-02-17T11:50:45.464 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529' 2026-02-17T11:50:45.464 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:45.894 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:50:45.894 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705711 2026-02-17T11:50:45.894 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705711 2026-02-17T11:50:45.895 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711' 2026-02-17T11:50:45.895 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:45.895 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:50:45.895 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574893 2026-02-17T11:50:45.895 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574893 2026-02-17T11:50:45.895 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-55834574893' 2026-02-17T11:50:45.895 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:45.895 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:50:45.895 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=459561500674 2026-02-17T11:50:45.895 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 459561500674 2026-02-17T11:50:45.895 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-55834574893 3-459561500674' 2026-02-17T11:50:45.895 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:45.896 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:50:45.896 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247849 2026-02-17T11:50:45.896 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247849 2026-02-17T11:50:45.896 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-55834574893 3-459561500674 4-98784247849' 2026-02-17T11:50:45.896 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:45.896 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:50:45.896 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=472446402562 2026-02-17T11:50:45.896 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 472446402562 2026-02-17T11:50:45.896 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-55834574893 3-459561500674 4-98784247849 5-472446402562' 2026-02-17T11:50:45.896 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:45.896 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:50:45.896 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986214 2026-02-17T11:50:45.897 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986214 2026-02-17T11:50:45.897 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-55834574893 3-459561500674 4-98784247849 5-472446402562 6-133143986214' 2026-02-17T11:50:45.897 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:45.897 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:50:45.962 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:50:45.962 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705712 2026-02-17T11:50:45.963 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705712 2026-02-17T11:50:45.963 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712' 2026-02-17T11:50:45.963 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:45.963 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:50:45.963 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574894 2026-02-17T11:50:45.963 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574894 2026-02-17T11:50:45.963 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712 2-55834574894' 2026-02-17T11:50:45.963 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:45.963 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:50:45.963 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=459561500675 2026-02-17T11:50:45.964 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 459561500675 2026-02-17T11:50:45.964 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712 2-55834574894 3-459561500675' 2026-02-17T11:50:45.964 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:45.964 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:50:45.964 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247851 2026-02-17T11:50:45.964 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247851 2026-02-17T11:50:45.964 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712 2-55834574894 3-459561500675 4-98784247851' 2026-02-17T11:50:45.964 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:45.964 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:50:45.964 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=472446402563 2026-02-17T11:50:45.964 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 472446402563 2026-02-17T11:50:45.964 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712 2-55834574894 3-459561500675 4-98784247851 5-472446402563' 2026-02-17T11:50:45.965 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:45.965 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:50:45.965 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986215 2026-02-17T11:50:45.965 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986215 2026-02-17T11:50:45.965 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712 2-55834574894 3-459561500675 4-98784247851 5-472446402563 6-133143986215' 2026-02-17T11:50:45.965 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:45.965 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822692 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822692 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-55834574893 3-459561500674 4-98784247849 5-472446402562 6-133143986214 7-154618822692' 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659171 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659171 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-55834574893 3-459561500674 4-98784247849 5-472446402562 6-133143986214 7-154618822692 8-176093659171' 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=433791696902 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 433791696902 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705711 2-55834574893 3-459561500674 4-98784247849 5-472446402562 6-133143986214 7-154618822692 8-176093659171 9-433791696902' 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836528 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:50:47.356 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836528 2026-02-17T11:50:47.357 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:47.357 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836528 2026-02-17T11:50:47.357 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836528' 2026-02-17T11:50:47.357 INFO:tasks.workunit.client.0.trial128.stderr:239230: waiting osd.0 seq 21474836528 2026-02-17T11:50:47.357 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:50:47.357 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836527 -lt 21474836528 2026-02-17T11:50:47.357 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:50:47.357 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:50:47.357 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:50:47.357 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836530 -lt 21474836528 2026-02-17T11:50:47.357 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:47.357 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705711 2026-02-17T11:50:47.357 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:47.357 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:50:47.357 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705711 2026-02-17T11:50:47.428 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ub_pg_stats: seq=154618822693 2026-02-17T11:50:47.429 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822693 2026-02-17T11:50:47.429 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712 2-55834574894 3-459561500675 4-98784247851 5-472446402563 6-133143986215 7-154618822693' 2026-02-17T11:50:47.429 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:47.429 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:50:47.429 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659172 2026-02-17T11:50:47.429 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659172 2026-02-17T11:50:47.429 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712 2-55834574894 3-459561500675 4-98784247851 5-472446402563 6-133143986215 7-154618822693 8-176093659172' 2026-02-17T11:50:47.429 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:50:47.429 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:50:47.429 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=433791696903 2026-02-17T11:50:47.430 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 433791696903 2026-02-17T11:50:47.430 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836529 1-38654705712 2-55834574894 3-459561500675 4-98784247851 5-472446402563 6-133143986215 7-154618822693 8-176093659172 9-433791696903' 2026-02-17T11:50:47.430 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:47.430 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836529 2026-02-17T11:50:47.430 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:47.430 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:50:47.430 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836529 2026-02-17T11:50:47.430 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:47.430 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836529 2026-02-17T11:50:47.430 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836529' 2026-02-17T11:50:47.430 INFO:tasks.workunit.client.0.trial128.stderr:239232: waiting osd.0 seq 21474836529 2026-02-17T11:50:47.430 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:50:47.431 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836527 -lt 21474836529 2026-02-17T11:50:47.431 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:50:47.431 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:50:47.431 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:50:47.431 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836530 -lt 21474836529 2026-02-17T11:50:47.431 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:47.431 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705712 2026-02-17T11:50:47.431 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:47.431 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:50:47.431 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705712 2026-02-17T11:50:47.838 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:47.839 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705711 2026-02-17T11:50:47.839 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705711' 2026-02-17T11:50:47.839 INFO:tasks.workunit.client.0.trial128.stderr:239230: waiting osd.1 seq 38654705711 2026-02-17T11:50:47.839 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:50:47.839 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705712 -lt 38654705711 2026-02-17T11:50:47.839 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:47.839 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574893 2026-02-17T11:50:47.839 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:47.839 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:50:47.839 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574893 2026-02-17T11:50:47.839 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:47.840 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574893 2026-02-17T11:50:47.840 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574893' 2026-02-17T11:50:47.840 INFO:tasks.workunit.client.0.trial128.stderr:239230: waiting osd.2 seq 55834574893 2026-02-17T11:50:47.840 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:50:47.840 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574894 -lt 55834574893 2026-02-17T11:50:47.840 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:47.840 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-459561500674 2026-02-17T11:50:47.840 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:47.840 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:50:47.840 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-459561500674 2026-02-17T11:50:47.840 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:47.840 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=459561500674 2026-02-17T11:50:47.841 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 459561500674' 2026-02-17T11:50:47.841 INFO:tasks.workunit.client.0.trial128.stderr:239230: waiting osd.3 seq 459561500674 2026-02-17T11:50:47.841 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:50:47.841 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 459561500675 -lt 459561500674 2026-02-17T11:50:47.841 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:47.841 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247849 2026-02-17T11:50:47.841 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:47.841 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:50:47.841 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247849 2026-02-17T11:50:47.841 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:47.841 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247849 2026-02-17T11:50:47.841 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247849' 2026-02-17T11:50:47.841 INFO:tasks.workunit.client.0.trial128.stderr:239230: waiting osd.4 seq 98784247849 2026-02-17T11:50:47.916 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.clientuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:47.916 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705712 2026-02-17T11:50:47.917 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705712' 2026-02-17T11:50:47.917 INFO:tasks.workunit.client.0.trial128.stderr:239232: waiting osd.1 seq 38654705712 2026-02-17T11:50:47.917 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:50:47.917 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705712 -lt 38654705712 2026-02-17T11:50:47.917 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:47.917 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574894 2026-02-17T11:50:47.917 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:47.917 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:50:47.917 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574894 2026-02-17T11:50:47.917 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:47.917 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574894 2026-02-17T11:50:47.917 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574894' 2026-02-17T11:50:47.918 INFO:tasks.workunit.client.0.trial128.stderr:239232: waiting osd.2 seq 55834574894 2026-02-17T11:50:47.918 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:50:47.918 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574894 -lt 55834574894 2026-02-17T11:50:47.918 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:47.918 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-459561500675 2026-02-17T11:50:47.918 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:47.918 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:50:47.918 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-459561500675 2026-02-17T11:50:47.918 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:47.918 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=459561500675 2026-02-17T11:50:47.918 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 459561500675' 2026-02-17T11:50:47.918 INFO:tasks.workunit.client.0.trial128.stderr:239232: waiting osd.3 seq 459561500675 2026-02-17T11:50:47.919 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:50:47.919 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 459561500675 -lt 459561500675 2026-02-17T11:50:47.919 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:47.919 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247851 2026-02-17T11:50:47.919 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:47.919 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:50:47.919 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247851 2026-02-17T11:50:47.919 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:47.919 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247851 2026-02-17T11:50:47.919 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247851' 2026-02-17T11:50:47.919 INFO:tasks.workunit.client.0.trial128.stderr:239232: waiting osd.4 seq 98784247851 2026-02-17T11:50:48.458 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:50:48.458 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247851 -lt 98784247849 2026-02-17T11:50:48.459 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:48.459 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-472446402562 2026-02-17T11:50:48.459 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:48.459 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:50:48.459 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-472446402562 2026-02-17T11:50:48.459 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:48.459 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=472446402562 2026-02-17T11:50:48.460 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 472446402562' 2026-02-17T11:50:48.460 INFO:tasks.workunit.client.0.trial128.stderr:239230: waiting osd.5 seq 472446402562 2026-02-17T11:50:48.460 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:50:48.460 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 472446402563 -lt 472446402562 2026-02-17T11:50:48.460 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:48.460 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986214 2026-02-17T11:50:48.460 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:48.461 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:50:48.461 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986214 2026-02-17T11:50:48.461 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:48.461 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986214 2026-02-17T11:50:48.461 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986214' 2026-02-17T11:50:48.461 INFO:tasks.workunit.client.0.trial128.stderr:239230: waiting osd.6 seq 133143986214 2026-02-17T11:50:48.461 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:50:48.462 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986215 -lt 133143986214 2026-02-17T11:50:48.462 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:48.462 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822692 2026-02-17T11:50:48.462 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:48.462 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:50:48.462 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822692 2026-02-17T11:50:48.462 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:48.463 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822692 2026-02-17T11:50:48.463 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822692' 2026-02-17T11:50:48.463 INFO:tasks.workunit.client.0.trial128.stderr:239230: waiting osd.7 seq 154618822692 2026-02-17T11:50:48.463 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:50:48.463 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822694 -lt 154618822692 2026-02-17T11:50:48.463 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:48.557 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:50:48.557 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247851 -lt 98784247851 2026-02-17T11:50:48.557 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:48.557 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-472446402563 2026-02-17T11:50:48.558 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:48.558 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:50:48.558 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-472446402563 2026-02-17T11:50:48.558 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:48.558 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=472446402563 2026-02-17T11:50:48.558 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 472446402563' 2026-02-17T11:50:48.558 INFO:tasks.workunit.client.0.trial128.stderr:239232: waiting osd.5 seq 472446402563 2026-02-17T11:50:48.559 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:50:48.559 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 472446402563 -lt 472446402563 2026-02-17T11:50:48.559 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:48.559 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986215 2026-02-17T11:50:48.559 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:48.559 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:50:48.559 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986215 2026-02-17T11:50:48.559 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:48.560 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986215 2026-02-17T11:50:48.560 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986215' 2026-02-17T11:50:48.560 INFO:tasks.workunit.client.0.trial128.stderr:239232: waiting osd.6 seq 133143986215 2026-02-17T11:50:48.560 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:50:48.560 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986216 -lt 133143986215 2026-02-17T11:50:48.560 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:48.560 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822693 2026-02-17T11:50:48.561 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:48.561 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:50:48.561 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822693 2026-02-17T11:50:48.561 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:48.561 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822693 2026-02-17T11:50:48.561 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822693' 2026-02-17T11:50:48.561 INFO:tasks.workunit.client.0.trial128.stderr:239232: waiting osd.7 seq 154618822693 2026-02-17T11:50:48.561 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:50:48.562 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822694 -lt 154618822693 2026-02-17T11:50:48.562 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:49.104 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659171 2026-02-17T11:50:49.105 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:49.105 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:50:49.105 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659171 2026-02-17T11:50:49.105 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:49.105 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659171 2026-02-17T11:50:49.105 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659171' 2026-02-17T11:50:49.105 INFO:tasks.workunit.client.0.trial128.stderr:239230: waiting osd.8 seq 176093659171 2026-02-17T11:50:49.105 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:50:49.105 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659172 -lt 176093659171 2026-02-17T11:50:49.105 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:49.105 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-433791696902 2026-02-17T11:50:49.105 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:49.106 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:50:49.106 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-433791696902 2026-02-17T11:50:49.106 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:49.106 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=433791696902 2026-02-17T11:50:49.106 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 433791696902' 2026-02-17T11:50:49.106 INFO:tasks.workunit.client.0.trial128.stderr:239230: waiting osd.9 seq 433791696902 2026-02-17T11:50:49.106 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:50:49.106 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 433791696904 -lt 433791696902 2026-02-17T11:50:49.106 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:50:49.106 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:50:49.106 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:50:49.106 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:50:49.106 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:50:49.106 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:50:49.107 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:50:49.107 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:50:49.107 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:50:49.107 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:50:49.107 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:50:49.107 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:50:49.107 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:50:49.231 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status_pg_stats: echo 8-176093659172 2026-02-17T11:50:49.231 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:49.232 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:50:49.232 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659172 2026-02-17T11:50:49.232 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:49.232 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659172 2026-02-17T11:50:49.232 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659172' 2026-02-17T11:50:49.232 INFO:tasks.workunit.client.0.trial128.stderr:239232: waiting osd.8 seq 176093659172 2026-02-17T11:50:49.232 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:50:49.232 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659172 -lt 176093659172 2026-02-17T11:50:49.232 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:50:49.232 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-433791696903 2026-02-17T11:50:49.233 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:50:49.233 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:50:49.233 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-433791696903 2026-02-17T11:50:49.233 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:50:49.233 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=433791696903 2026-02-17T11:50:49.233 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 433791696903' 2026-02-17T11:50:49.233 INFO:tasks.workunit.client.0.trial128.stderr:239232: waiting osd.9 seq 433791696903 2026-02-17T11:50:49.233 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:50:49.233 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 433791696904 -lt 433791696903 2026-02-17T11:50:49.233 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:50:49.233 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:50:49.233 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:50:49.233 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:50:49.233 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:50:49.233 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:50:49.234 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:50:49.234 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:50:49.234 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:50:49.234 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:50:49.234 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:50:49.234 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:50:49.234 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:50:49.283 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:50:49.284 INFO:tasks.workunit.client.0.trial128.stderr:239230: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:50:49.284 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:50:49.284 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:50:49.284 INFO:tasks.workunit.client.0.trial128.stderr:239230: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:50:49.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:50:49.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:50:49.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 239228 2026-02-17T11:50:49.422 INFO:tasks.workunit.client.0.trial128.stderr: 2026-02-17T11:50:49.422 INFO:tasks.workunit.client.0.trial128.stderr:239232: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:50:49.423 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:50:49.423 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:50:49.423 INFO:tasks.workunit.client.0.trial128.stderr:239232: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:50:49.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:50:49.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T11:50:49.423 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T11:50:49.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T11:50:49.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:197: corrupt_and_repair_two: return_code=0 2026-02-17T11:50:49.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:198: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T11:50:49.424 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: get_pg ecpool SOMETHING 2026-02-17T11:50:49.424 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:50:49.425 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:50:49.425 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T11:50:49.425 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:50:49.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:203: corrupt_and_repair_two: local pg=2.0 2026-02-17T11:50:49.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:204: corrupt_and_repair_two: repair 2.0 2026-02-17T11:50:49.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T11:50:49.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T11:50:49.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:50:49.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:50:49.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:50:49.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:50:49.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:50:20.429951+0000 2026-02-17T11:50:49.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T11:50:49.872 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.3 to repair 2026-02-17T11:50:49.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T11:50:20.429951+0000 2026-02-17T11:50:49.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T11:50:49.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:50:20.429951+0000 2026-02-17T11:50:49.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:50:49.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:50:49.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:50:49.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:50:49.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:50:49.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:50:49.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:50:49.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:50:50.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:50:20.429951+0000 '>' 2026-02-17T11:50:20.429951+0000 2026-02-17T11:50:50.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:50:51.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:50:51.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:50:51.043 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:50:51.043 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:50:51.043 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:50:51.043 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:50:51.043 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:50:51.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:50:20.429951+0000 '>' 2026-02-17T11:50:20.429951+0000 2026-02-17T11:50:51.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:50:52.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:50:52.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:50:52.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:50:52.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:50:52.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:50:52.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:50:52.198 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:50:52.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:50:20.429951+0000 '>' 2026-02-17T11:50:20.429951+0000 2026-02-17T11:50:52.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:50:53.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:50:53.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:50:53.351 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:50:53.351 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:50:53.351 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:50:53.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:50:53.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:50:53.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:50:20.429951+0000 '>' 2026-02-17T11:50:20.429951+0000 2026-02-17T11:50:53.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:50:54.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:50:54.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:50:54.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:50:54.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:50:54.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:50:54.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:50:54.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:50:54.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:50:20.429951+0000 '>' 2026-02-17T11:50:20.429951+0000 2026-02-17T11:50:54.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:50:55.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:50:55.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:50:55.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:50:55.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:50:55.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:50:55.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:50:55.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:50:55.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:50:20.429951+0000 '>' 2026-02-17T11:50:20.429951+0000 2026-02-17T11:50:55.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:50:56.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:50:56.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:50:56.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T11:50:56.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T11:50:56.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:50:56.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:50:56.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T11:50:56.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:50:50.458543+0000 '>' 2026-02-17T11:50:20.429951+0000 2026-02-17T11:50:56.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:50:56.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:208: corrupt_and_repair_two: pids= 2026-02-17T11:50:56.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:209: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:50:56.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:50:56.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:50:56.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 242834"' 2026-02-17T11:50:56.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 242834' 2026-02-17T11:50:56.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:210: corrupt_and_repair_two: run_in_background pids objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T11:50:56.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T11:50:56.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T11:50:56.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 242835"' 2026-02-17T11:50:56.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 242835' 2026-02-17T11:50:56.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:211: corrupt_and_repair_two: wait_background pids 2026-02-17T11:50:56.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 242834 242835' 2026-02-17T11:50:56.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T11:50:56.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:50:56.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 242834 2026-02-17T11:50:56.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:50:56.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/242837: /' 2026-02-17T11:50:56.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T11:50:56.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/242839: /' 2026-02-17T11:50:59.126 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:50:59.127 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:50:59.127 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T11:50:59.127 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:50:59.127 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:50:59.127 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:50:59.127 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:50:59.127 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T11:50:59.127 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:50:59.127 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T11:50:59.128 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:50:59.128 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:50:59.128 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:50:59.128 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:50:59.128 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:50:59.128 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:50:59.128 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING list-attrs 2026-02-17T11:50:59.128 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:50:59.128 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:50:59.129 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T11:50:59.129 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:50:59.129 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:50:59.129 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING list-attrs 2026-02-17T11:50:59.129 INFO:tasks.workunit.client.0.trial128.stderr:242837: Error getting attr on : 2.0s0_head,0#-4:00000000:::scrub_2.0s0:head#, (61) No data available 2026-02-17T11:50:59.129 INFO:tasks.workunit.client.0.trial128.stderr:242837: _ 2026-02-17T11:50:59.129 INFO:tasks.workunit.client.0.trial128.stderr:242837: hinfo_key 2026-02-17T11:50:59.129 INFO:tasks.workunit.client.0.trial128.stderr:242837: snapset 2026-02-17T11:50:59.129 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T11:50:59.129 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:50:59.130 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:50:59.130 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T11:50:59.130 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:50:59.130 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T11:50:59.130 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:50:59.131 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:50:59.131 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:50:59.131 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:50:59.131 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T11:50:59.131 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T11:50:59.131 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:50:59.131 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:50:59.131 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:50:59.131 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:50:59.132 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:50:59.132 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:50:59.132 INFO:tasks.workunit.client.0.trial128.stderr:242837: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:50:59.132 INFO:tasks.workunit.client.0.trial128.stderr:242837: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:50:59.132 INFO:tasks.workunit.client.0.trial128.stderr:242837: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:50:59.132 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:50:59.132 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:50:59.132 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:50:59.132 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:50:59.132 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:50:59.133 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:50:59.133 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:50:59.133 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:50:59.133 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:50:59.133 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:50:59.133 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:50:59.133 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T11:50:59.133 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T11:50:59.133 INFO:tasks.workunit.client.0.trial128.stderr:242837: start osd.3 2026-02-17T11:50:59.136 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:50:59.136 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:50:59.136 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=5 2026-02-17T11:50:59.136 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:50:59.136 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T11:50:59.136 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:50:59.136 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:50:59.136 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=5 2026-02-17T11:50:59.136 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:50:59.137 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.5 2026-02-17T11:50:59.137 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:50:59.137 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:50:59.137 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:50:59.137 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:50:59.137 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:50:59.137 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:50:59.137 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 5 SOMETHING list-attrs 2026-02-17T11:50:59.137 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:50:59.137 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:50:59.137 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=5 2026-02-17T11:50:59.137 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:50:59.137 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:50:59.138 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/5 SOMETHING list-attrs 2026-02-17T11:50:59.138 INFO:tasks.workunit.client.0.trial128.stderr:242839: _ 2026-02-17T11:50:59.138 INFO:tasks.workunit.client.0.trial128.stderr:242839: hinfo_key 2026-02-17T11:50:59.138 INFO:tasks.workunit.client.0.trial128.stderr:242839: snapset 2026-02-17T11:50:59.138 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 5 2026-02-17T11:50:59.138 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:50:59.138 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:50:59.138 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=5 2026-02-17T11:50:59.138 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:50:59.138 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/5 2026-02-17T11:50:59.138 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:50:59.138 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:50:59.139 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:50:59.139 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:50:59.139 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/5' 2026-02-17T11:50:59.139 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/5/journal' 2026-02-17T11:50:59.140 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:50:59.140 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:50:59.140 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:50:59.140 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:50:59.140 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:50:59.140 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:50:59.140 INFO:tasks.workunit.client.0.trial128.stderr:242839: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:50:59.140 INFO:tasks.workunit.client.0.trial128.stderr:242839: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:50:59.140 INFO:tasks.workunit.client.0.trial128.stderr:242839: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:50:59.140 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:50:59.140 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:50:59.140 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:50:59.140 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:50:59.141 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:50:59.141 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:50:59.141 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:50:59.141 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:50:59.141 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:50:59.141 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:50:59.141 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:50:59.141 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/5 2026-02-17T11:50:59.141 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.5 2026-02-17T11:50:59.141 INFO:tasks.workunit.client.0.trial128.stderr:242839: start osd.5 2026-02-17T11:51:02.916 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 5 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/5 --osd-journal=td/osd-scrub-repair/5/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_reclen=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:51:02.916 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T11:51:02.916 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T11:51:02.916 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:51:02.916 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:51:02.916 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:51:02.916 INFO:tasks.workunit.client.0.trial128.stderr:242837: 2026-02-17T11:50:59.144+0000 7f810630c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:02.916 INFO:tasks.workunit.client.0.trial128.stderr:242837: 2026-02-17T11:50:59.144+0000 7f810630c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:02.917 INFO:tasks.workunit.client.0.trial128.stderr:242837: 2026-02-17T11:50:59.144+0000 7f810630c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:02.917 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T11:51:02.917 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:51:02.917 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T11:51:02.917 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:51:02.917 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:51:02.917 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:02.917 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:51:02.917 INFO:tasks.workunit.client.0.trial128.stderr:242837: 0 2026-02-17T11:51:02.917 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:02.917 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:51:02.918 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:02.918 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:02.918 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:02.918 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:51:02.918 INFO:tasks.workunit.client.0.trial128.stderr:242837: 1 2026-02-17T11:51:02.918 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:02.918 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:51:02.918 INFO:tasks.workunit.client.0.trial128.stderr:242837: 2026-02-17T11:51:00.480+0000 7f810630c8c0 -1 Falling back to public interface 2026-02-17T11:51:02.918 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:02.918 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:02.918 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:02.918 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:51:02.919 INFO:tasks.workunit.client.0.trial128.stderr:242837: 2 2026-02-17T11:51:02.919 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:02.919 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:51:02.919 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:02.919 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:02.919 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:02.919 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:51:02.919 INFO:tasks.workunit.client.0.trial128.stderr:242837: 3 2026-02-17T11:51:02.919 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:02.919 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:51:03.413 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.clientovery_ops 2026-02-17T11:51:03.413 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/5/whoami 2026-02-17T11:51:03.413 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 5 = 5 ']' 2026-02-17T11:51:03.413 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:51:03.413 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:51:03.414 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:51:03.414 INFO:tasks.workunit.client.0.trial128.stderr:242839: 2026-02-17T11:50:59.144+0000 7f7a80c418c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:03.414 INFO:tasks.workunit.client.0.trial128.stderr:242839: 2026-02-17T11:50:59.144+0000 7f7a80c418c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:03.414 INFO:tasks.workunit.client.0.trial128.stderr:242839: 2026-02-17T11:50:59.144+0000 7f7a80c418c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:03.414 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 5 2026-02-17T11:51:03.414 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:51:03.414 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=5 2026-02-17T11:51:03.414 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:51:03.414 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:51:03.414 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:03.414 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:51:03.414 INFO:tasks.workunit.client.0.trial128.stderr:242839: 0 2026-02-17T11:51:03.414 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:03.414 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:51:03.415 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:03.415 INFO:tasks.workunit.client.0.trial128.stderr:242839: 2026-02-17T11:51:00.412+0000 7f7a80c418c0 -1 Falling back to public interface 2026-02-17T11:51:03.415 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:03.415 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:03.415 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:51:03.415 INFO:tasks.workunit.client.0.trial128.stderr:242839: 1 2026-02-17T11:51:03.415 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:03.415 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:51:03.415 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:03.415 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:03.415 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:03.415 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:51:03.415 INFO:tasks.workunit.client.0.trial128.stderr:242839: 2 2026-02-17T11:51:03.415 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:03.416 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:51:03.416 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:03.416 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:03.416 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:03.416 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:51:03.416 INFO:tasks.workunit.client.0.trial128.stderr:242839: 3 2026-02-17T11:51:03.416 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:03.416 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:51:03.416 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:04.397 INFO:tasks.workunit.client.0.trial128.stderr:242839: 2026-02-17T11:51:03.400+0000 7f7a80c418c0 -1 osd.5 111 log_to_monitors true 2026-02-17T11:51:04.397 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:04.397 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:04.397 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:51:04.397 INFO:tasks.workunit.client.0.trial128.stderr:242839: 4 2026-02-17T11:51:04.397 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:04.397 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.5 up' 2026-02-17T11:51:04.398 INFO:tasks.workunit.client.0.trial128.stderr:242839: osd.5 up in weight 1 up_from 116 up_thru 116 down_at 112 last_clean_interval [110,111) [v2:127.0.0.1:6826/918738473,v1:127.0.0.1:6827/918738473] [v2:127.0.0.1:6828/918738473,v1:127.0.0.1:6829/918738473] exists,up 80987ebb-2096-4a73-b7c5-64048030732e 2026-02-17T11:51:04.398 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:51:04.398 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:51:04.398 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:51:04.398 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:51:04.398 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:51:04.398 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:51:04.398 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:51:04.398 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:51:04.398 INFO:tasks.workunit.client.0.trial128.stderr:242839: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:51:04.398 INFO:tasks.workunit.client.0.trial128.stderr:242839: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:51:04.398 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:51:04.399 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:51:04.399 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:51:04.399 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:51:04.399 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:51:04.399 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:51:04.399 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:51:04.399 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:51:04.399 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:51:04.399 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:51:04.399 INFO:tasks.workunit.client.0.trial128.stderr:242839: 1 2026-02-17T11:51:04.399 INFO:tasks.workunit.client.0.trial128.stderr:242839: 2 2026-02-17T11:51:04.399 INFO:tasks.workunit.client.0.trial128.stderr:242839: 3 2026-02-17T11:51:04.400 INFO:tasks.workunit.client.0.trial128.stderr:242839: 4 2026-02-17T11:51:04.400 INFO:tasks.workunit.client.0.trial128.stderr:242839: 5 2026-02-17T11:51:04.400 INFO:tasks.workunit.client.0.trial128.stderr:242839: 6 2026-02-17T11:51:04.400 INFO:tasks.workunit.client.0.trial128.stderr:242839: 7 2026-02-17T11:51:04.400 INFO:tasks.workunit.client.0.trial128.stderr:242839: 8 2026-02-17T11:51:04.400 INFO:tasks.workunit.client.0.trial128.stderr:242839: 9' 2026-02-17T11:51:04.400 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:51:04.400 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:04.400 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:51:04.400 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836534 2026-02-17T11:51:04.400 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836534 2026-02-17T11:51:04.400 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534' 2026-02-17T11:51:04.403 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:04.403 INFO:tasks.workunit.client.0.trial128.stderr:242837: 2026-02-17T11:51:03.580+0000 7f810630c8c0 -1 osd.3 111 log_to_monitors true 2026-02-17T11:51:04.403 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:04.404 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:04.404 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:51:04.404 INFO:tasks.workunit.client.0.trial128.stderr:242837: 4 2026-02-17T11:51:04.404 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:04.404 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T11:51:04.404 INFO:tasks.workunit.client.0.trial128.stderr:242837: osd.3 up in weight 1 up_from 118 up_thru 118 down_at 112 last_clean_interval [107,111) [v2:127.0.0.1:6842/3391669157,v1:127.0.0.1:6843/3391669157] [v2:127.0.0.1:6844/3391669157,v1:127.0.0.1:6845/3391669157] exists,up 063dd7f2-c316-40f3-bf48-6933c998a894 2026-02-17T11:51:04.404 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:51:04.404 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:51:04.404 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:51:04.404 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:51:04.405 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:51:04.405 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:51:04.405 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:51:04.405 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:51:04.405 INFO:tasks.workunit.client.0.trial128.stderr:242837: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:51:04.405 INFO:tasks.workunit.client.0.trial128.stderr:242837: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:51:04.405 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:51:04.405 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:51:04.405 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:51:04.405 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:51:04.405 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:51:04.405 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:51:04.406 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:51:04.406 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:51:04.406 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:51:04.406 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:51:04.406 INFO:tasks.workunit.client.0.trial128.stderr:242837: 1 2026-02-17T11:51:04.406 INFO:tasks.workunit.client.0.trial128.stderr:242837: 2 2026-02-17T11:51:04.406 INFO:tasks.workunit.client.0.trial128.stderr:242837: 3 2026-02-17T11:51:04.406 INFO:tasks.workunit.client.0.trial128.stderr:242837: 4 2026-02-17T11:51:04.406 INFO:tasks.workunit.client.0.trial128.stderr:242837: 5 2026-02-17T11:51:04.406 INFO:tasks.workunit.client.0.trial128.stderr:242837: 6 2026-02-17T11:51:04.406 INFO:tasks.workunit.client.0.trial128.stderr:242837: 7 2026-02-17T11:51:04.406 INFO:tasks.workunit.client.0.trial128.stderr:242837: 8 2026-02-17T11:51:04.406 INFO:tasks.workunit.client.0.trial128.stderr:242837: 9' 2026-02-17T11:51:04.407 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:51:04.407 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:04.407 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:51:04.407 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836535 2026-02-17T11:51:04.407 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836535 2026-02-17T11:51:04.407 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535' 2026-02-17T11:51:04.838 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:04.839 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:51:04.839 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705718 2026-02-17T11:51:04.839 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705718 2026-02-17T11:51:04.839 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705718' 2026-02-17T11:51:04.839 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:04.839 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:51:04.839 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574900 2026-02-17T11:51:04.839 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574900 2026-02-17T11:51:04.839 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705718 2-55834574900' 2026-02-17T11:51:04.839 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:04.840 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:51:04.840 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=506806140931 2026-02-17T11:51:04.840 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 506806140931 2026-02-17T11:51:04.840 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705718 2-55834574900 3-506806140931' 2026-02-17T11:51:04.840 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:04.840 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:51:04.840 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247856 2026-02-17T11:51:04.840 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247856 2026-02-17T11:51:04.840 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705718 2-55834574900 3-506806140931 4-98784247856' 2026-02-17T11:51:04.840 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:04.840 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:51:04.841 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=498216206339 2026-02-17T11:51:04.841 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 498216206339 2026-02-17T11:51:04.841 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705718 2-55834574900 3-506806140931 4-98784247856 5-498216206339' 2026-02-17T11:51:04.841 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:04.841 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:51:04.841 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986221 2026-02-17T11:51:04.841 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986221 2026-02-17T11:51:04.841 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705718 2-55834574900 3-506806140931 4-98784247856 5-498216206339 6-133143986221' 2026-02-17T11:51:04.841 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:04.904 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph ids 2026-02-17T11:51:04.904 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:51:04.904 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705717 2026-02-17T11:51:04.904 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705717 2026-02-17T11:51:04.904 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705717' 2026-02-17T11:51:04.905 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:04.905 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:51:04.905 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574899 2026-02-17T11:51:04.905 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574899 2026-02-17T11:51:04.905 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705717 2-55834574899' 2026-02-17T11:51:04.905 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:04.905 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T11:51:04.905 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=506806140930 2026-02-17T11:51:04.905 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 506806140930 2026-02-17T11:51:04.905 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705717 2-55834574899 3-506806140930' 2026-02-17T11:51:04.905 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:04.905 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-17T11:51:04.906 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247855 2026-02-17T11:51:04.906 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247855 2026-02-17T11:51:04.906 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705717 2-55834574899 3-506806140930 4-98784247855' 2026-02-17T11:51:04.906 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:04.906 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-17T11:51:04.906 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=498216206338 2026-02-17T11:51:04.906 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 498216206338 2026-02-17T11:51:04.906 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705717 2-55834574899 3-506806140930 4-98784247855 5-498216206338' 2026-02-17T11:51:04.906 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:04.907 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-17T11:51:04.907 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986220 2026-02-17T11:51:04.907 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986220 2026-02-17T11:51:04.907 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705717 2-55834574899 3-506806140930 4-98784247855 5-498216206338 6-133143986220' 2026-02-17T11:51:04.907 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:04.907 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822698 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822698 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705717 2-55834574899 3-506806140930 4-98784247855 5-498216206338 6-133143986220 7-154618822698' 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659176 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659176 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705717 2-55834574899 3-506806140930 4-98784247855 5-498216206338 6-133143986220 7-154618822698 8-176093659176' 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=433791696908 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 433791696908 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836534 1-38654705717 2-55834574899 3-506806140930 4-98784247855 5-498216206338 6-133143986220 7-154618822698 8-176093659176 9-433791696908' 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836534 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836534 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836534 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836534' 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: waiting osd.0 seq 21474836534 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836533 -lt 21474836534 2026-02-17T11:51:06.366 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:51:06.367 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:51:06.367 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:51:06.367 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836536 -lt 21474836534 2026-02-17T11:51:06.367 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:06.367 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705717 2026-02-17T11:51:06.367 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:06.367 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:51:06.367 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705717 2026-02-17T11:51:06.369 INFO:tasks.workunit.client.0.trial128.stderr:242839: //tell osd.7 flush_pg_stats 2026-02-17T11:51:06.369 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822699 2026-02-17T11:51:06.369 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822699 2026-02-17T11:51:06.369 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705718 2-55834574900 3-506806140931 4-98784247856 5-498216206339 6-133143986221 7-154618822699' 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659177 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659177 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705718 2-55834574900 3-506806140931 4-98784247856 5-498216206339 6-133143986221 7-154618822699 8-176093659177' 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=433791696909 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 433791696909 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705718 2-55834574900 3-506806140931 4-98784247856 5-498216206339 6-133143986221 7-154618822699 8-176093659177 9-433791696909' 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836535 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836535 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836535 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836535' 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: waiting osd.0 seq 21474836535 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836533 -lt 21474836535 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836536 -lt 21474836535 2026-02-17T11:51:06.370 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:06.371 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705718 2026-02-17T11:51:06.371 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:06.371 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:51:06.849 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705718 2026-02-17T11:51:06.849 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:06.849 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705718 2026-02-17T11:51:06.849 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705718' 2026-02-17T11:51:06.849 INFO:tasks.workunit.client.0.trial128.stderr:242837: waiting osd.1 seq 38654705718 2026-02-17T11:51:06.849 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:51:06.849 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705718 -lt 38654705718 2026-02-17T11:51:06.849 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:06.849 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574900 2026-02-17T11:51:06.849 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:06.849 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:51:06.849 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574900 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574900 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574900' 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: waiting osd.2 seq 55834574900 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574900 -lt 55834574900 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-506806140931 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-506806140931 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=506806140931 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 506806140931' 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: waiting osd.3 seq 506806140931 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 506806140931 -lt 506806140931 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247856 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247856 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247856 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705717 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705717' 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242839: waiting osd.1 seq 38654705717 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705718 -lt 38654705717 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574899 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574899 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574899 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574899' 2026-02-17T11:51:06.850 INFO:tasks.workunit.client.0.trial128.stderr:242839: waiting osd.2 seq 55834574899 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574900 -lt 55834574899 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-506806140930 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-506806140930 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=506806140930 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 506806140930' 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: waiting osd.3 seq 506806140930 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 506806140931 -lt 506806140930 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 4-98784247855 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=4 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 4-98784247855 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247855 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.4 seq 98784247855' 2026-02-17T11:51:06.851 INFO:tasks.workunit.client.0.trial128.stderr:242839: waiting osd.4 seq 98784247855 2026-02-17T11:51:07.482 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clonewaiting osd.4 seq 98784247856' 2026-02-17T11:51:07.482 INFO:tasks.workunit.client.0.trial128.stderr:242837: waiting osd.4 seq 98784247856 2026-02-17T11:51:07.482 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:51:07.482 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247857 -lt 98784247856 2026-02-17T11:51:07.482 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:07.483 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-498216206339 2026-02-17T11:51:07.483 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:07.483 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:51:07.483 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-498216206339 2026-02-17T11:51:07.483 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:07.483 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=498216206339 2026-02-17T11:51:07.483 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 498216206339' 2026-02-17T11:51:07.483 INFO:tasks.workunit.client.0.trial128.stderr:242837: waiting osd.5 seq 498216206339 2026-02-17T11:51:07.483 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:51:07.483 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 498216206339 -lt 498216206339 2026-02-17T11:51:07.484 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:07.484 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986221 2026-02-17T11:51:07.484 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:07.484 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:51:07.484 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986221 2026-02-17T11:51:07.484 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:07.484 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986221 2026-02-17T11:51:07.485 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986221' 2026-02-17T11:51:07.485 INFO:tasks.workunit.client.0.trial128.stderr:242837: waiting osd.6 seq 133143986221 2026-02-17T11:51:07.485 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:51:07.485 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986221 -lt 133143986221 2026-02-17T11:51:07.485 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:07.485 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822699 2026-02-17T11:51:07.486 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:07.486 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:51:07.486 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822699 2026-02-17T11:51:07.486 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:07.486 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822699 2026-02-17T11:51:07.486 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822699' 2026-02-17T11:51:07.486 INFO:tasks.workunit.client.0.trial128.stderr:242837: waiting osd.7 seq 154618822699 2026-02-17T11:51:07.486 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:51:07.487 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822699 -lt 154618822699 2026-02-17T11:51:07.492 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_p.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-17T11:51:07.492 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247857 -lt 98784247855 2026-02-17T11:51:07.492 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:07.492 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 5-498216206338 2026-02-17T11:51:07.492 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:07.492 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=5 2026-02-17T11:51:07.492 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 5-498216206338 2026-02-17T11:51:07.493 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:07.493 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=498216206338 2026-02-17T11:51:07.493 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.5 seq 498216206338' 2026-02-17T11:51:07.493 INFO:tasks.workunit.client.0.trial128.stderr:242839: waiting osd.5 seq 498216206338 2026-02-17T11:51:07.493 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-17T11:51:07.493 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 498216206339 -lt 498216206338 2026-02-17T11:51:07.493 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:07.493 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 6-133143986220 2026-02-17T11:51:07.493 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:07.493 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=6 2026-02-17T11:51:07.493 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 6-133143986220 2026-02-17T11:51:07.493 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:07.494 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986220 2026-02-17T11:51:07.494 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.6 seq 133143986220' 2026-02-17T11:51:07.494 INFO:tasks.workunit.client.0.trial128.stderr:242839: waiting osd.6 seq 133143986220 2026-02-17T11:51:07.494 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-17T11:51:07.494 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986221 -lt 133143986220 2026-02-17T11:51:07.494 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:07.494 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 7-154618822698 2026-02-17T11:51:07.494 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:07.494 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=7 2026-02-17T11:51:07.494 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 7-154618822698 2026-02-17T11:51:07.494 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:07.494 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822698 2026-02-17T11:51:07.494 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.7 seq 154618822698' 2026-02-17T11:51:07.495 INFO:tasks.workunit.client.0.trial128.stderr:242839: waiting osd.7 seq 154618822698 2026-02-17T11:51:07.495 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-17T11:51:07.495 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822699 -lt 154618822698 2026-02-17T11:51:07.495 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:08.133 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274g_stats: for s in $seqs 2026-02-17T11:51:08.134 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 8-176093659177 2026-02-17T11:51:08.134 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:08.134 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:51:08.134 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659177 2026-02-17T11:51:08.134 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:08.134 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659177 2026-02-17T11:51:08.134 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659177' 2026-02-17T11:51:08.134 INFO:tasks.workunit.client.0.trial128.stderr:242837: waiting osd.8 seq 176093659177 2026-02-17T11:51:08.135 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:51:08.135 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659178 -lt 176093659177 2026-02-17T11:51:08.135 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:08.135 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-433791696909 2026-02-17T11:51:08.135 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:08.135 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:51:08.135 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-433791696909 2026-02-17T11:51:08.135 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:08.135 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=433791696909 2026-02-17T11:51:08.135 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 433791696909' 2026-02-17T11:51:08.135 INFO:tasks.workunit.client.0.trial128.stderr:242837: waiting osd.9 seq 433791696909 2026-02-17T11:51:08.135 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:51:08.136 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 433791696909 -lt 433791696909 2026-02-17T11:51:08.136 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:51:08.136 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:51:08.136 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:51:08.136 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:51:08.136 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:51:08.136 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:51:08.136 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:51:08.136 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:51:08.136 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:51:08.137 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:51:08.137 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:51:08.137 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:51:08.137 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:51:08.158 INFO:tasks.workunit.client.0.trial128.stderr:242837: /: flush_pg_stats: echo 8-176093659176 2026-02-17T11:51:08.159 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:08.159 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=8 2026-02-17T11:51:08.159 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 8-176093659176 2026-02-17T11:51:08.159 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:08.159 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659176 2026-02-17T11:51:08.159 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.8 seq 176093659176' 2026-02-17T11:51:08.159 INFO:tasks.workunit.client.0.trial128.stderr:242839: waiting osd.8 seq 176093659176 2026-02-17T11:51:08.159 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-17T11:51:08.159 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659178 -lt 176093659176 2026-02-17T11:51:08.159 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:08.160 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 9-433791696908 2026-02-17T11:51:08.160 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:08.160 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=9 2026-02-17T11:51:08.160 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 9-433791696908 2026-02-17T11:51:08.160 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:08.160 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=433791696908 2026-02-17T11:51:08.160 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.9 seq 433791696908' 2026-02-17T11:51:08.160 INFO:tasks.workunit.client.0.trial128.stderr:242839: waiting osd.9 seq 433791696908 2026-02-17T11:51:08.160 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-17T11:51:08.160 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 433791696909 -lt 433791696908 2026-02-17T11:51:08.160 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:51:08.160 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:51:08.160 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:51:08.161 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T11:51:08.161 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:51:08.161 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:51:08.161 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:51:08.161 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:51:08.161 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:51:08.161 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:51:08.161 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:51:08.161 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T11:51:08.161 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:51:08.323 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:51:08.323 INFO:tasks.workunit.client.0.trial128.stderr:242837: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:51:08.324 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:51:08.324 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:51:08.324 INFO:tasks.workunit.client.0.trial128.stderr:242837: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:51:08.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:51:08.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T11:51:08.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 242835 2026-02-17T11:51:08.339 INFO:tasks.workunit.client.0.trial128.stderr: status 2026-02-17T11:51:08.339 INFO:tasks.workunit.client.0.trial128.stderr:242839: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:51:08.340 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T11:51:08.340 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:51:08.340 INFO:tasks.workunit.client.0.trial128.stderr:242839: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:51:08.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T11:51:08.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T11:51:08.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T11:51:08.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T11:51:08.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:212: corrupt_and_repair_two: return_code=0 2026-02-17T11:51:08.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:213: corrupt_and_repair_two: '[' 0 -ne 0 ']' 2026-02-17T11:51:08.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:215: corrupt_and_repair_two: rados --pool ecpool get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:51:08.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:216: corrupt_and_repair_two: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:51:08.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T11:51:08.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:51:08.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:51:08.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:51:08.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:51:08.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:51:08.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:51:08.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:51:08.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:51:08.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:51:08.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:51:08.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:51:08.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:51:08.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:51:08.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:51:08.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:51:08.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:51:08.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:51:08.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:51:08.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:51:08.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:51:08.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:51:08.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:51:08.726 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:51:08.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:51:08.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:51:08.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:51:08.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:51:08.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:51:08.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T11:51:08.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T11:51:08.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T11:51:08.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T11:51:08.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:51:08.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:51:08.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:51:08.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:51:08.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:51:08.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:51:08.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:51:08.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:51:08.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:51:08.732 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:51:08.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:51:08.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:51:08.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:51:08.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:51:08.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:51:08.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:51:08.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:51:08.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:51:08.737 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:51:08.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:51:08.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:51:08.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:51:08.741 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:51:08.741 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:51:08.741 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:51:08.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:51:08.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:51:08.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:51:08.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T11:51:08.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:51:08.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:51:08.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:51:08.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T11:51:08.746 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:51:08.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T11:51:08.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:51:08.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T11:51:08.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_and_repair_replicated td/osd-scrub-repair 2026-02-17T11:51:08.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:94: TEST_corrupt_and_repair_replicated: local dir=td/osd-scrub-repair 2026-02-17T11:51:08.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:95: TEST_corrupt_and_repair_replicated: local poolname=rbd 2026-02-17T11:51:08.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:97: TEST_corrupt_and_repair_replicated: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-17T11:51:08.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T11:51:08.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T11:51:08.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T11:51:08.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T11:51:08.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T11:51:08.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-17T11:51:08.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T11:51:08.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:51:08.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:51:08.770 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:51:08.770 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:51:08.770 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:51:08.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:51:08.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T11:51:08.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T11:51:08.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T11:51:08.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:51:08.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:51:08.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T11:51:08.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T11:51:08.802 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:51:08.802 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:51:08.803 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:51:08.804 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:51:08.804 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:51:08.804 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:51:08.804 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:51:08.804 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:51:08.804 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T11:51:08.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T11:51:08.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:51:08.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:51:08.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T11:51:08.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T11:51:08.861 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:51:08.861 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:51:08.861 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:51:08.861 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:51:08.861 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:51:08.861 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:51:08.861 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:51:08.861 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:51:08.861 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T11:51:08.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:98: TEST_corrupt_and_repair_replicated: run_mgr td/osd-scrub-repair x 2026-02-17T11:51:08.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T11:51:08.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T11:51:08.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T11:51:08.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T11:51:08.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T11:51:08.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T11:51:09.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T11:51:09.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:51:09.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:51:09.010 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:51:09.010 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:51:09.010 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:51:09.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:51:09.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:51:09.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:51:09.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:99: TEST_corrupt_and_repair_replicated: run_osd td/osd-scrub-repair 0 2026-02-17T11:51:09.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:51:09.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:51:09.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T11:51:09.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:51:09.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:51:09.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:51:09.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:51:09.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:51:09.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:51:09.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:51:09.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:51:09.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:51:09.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:51:09.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:51:09.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:51:09.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:51:09.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:51:09.034 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:51:09.034 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:51:09.034 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:51:09.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:51:09.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:51:09.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:51:09.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:51:09.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:51:09.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:51:09.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:51:09.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:51:09.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:51:09.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:51:09.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:51:09.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:51:09.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:51:09.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:51:09.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:51:09.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=66c875c2-c236-40bb-9241-94fcfe167999 2026-02-17T11:51:09.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 66c875c2-c236-40bb-9241-94fcfe167999' 2026-02-17T11:51:09.037 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 66c875c2-c236-40bb-9241-94fcfe167999 2026-02-17T11:51:09.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:51:09.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAtVpRp18VAAhAAdtRyDZAWux3ZORa2YjfDxQ== 2026-02-17T11:51:09.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAtVpRp18VAAhAAdtRyDZAWux3ZORa2YjfDxQ=="}' 2026-02-17T11:51:09.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 66c875c2-c236-40bb-9241-94fcfe167999 -i td/osd-scrub-repair/0/new.json 2026-02-17T11:51:09.138 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:51:09.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T11:51:09.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAtVpRp18VAAhAAdtRyDZAWux3ZORa2YjfDxQ== --osd-uuid 66c875c2-c236-40bb-9241-94fcfe167999 2026-02-17T11:51:09.163 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:09.152+0000 7effe3dd98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:09.164 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:09.152+0000 7effe3dd98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:09.165 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:09.156+0000 7effe3dd98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:09.166 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:09.156+0000 7effe3dd98c0 -1 bdev(0x5630ce989c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:51:09.166 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:09.156+0000 7effe3dd98c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T11:51:11.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T11:51:11.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:51:11.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T11:51:11.509 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:51:11.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:51:11.701 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:51:11.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T11:51:11.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:51:11.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:51:11.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:51:11.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:51:11.723 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:11.712+0000 7f4e4f0cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:11.724 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:11.712+0000 7f4e4f0cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:11.725 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:11.716+0000 7f4e4f0cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:11.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T11:51:11.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:51:11.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:51:11.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:51:11.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:51:11.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:11.858 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:51:11.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:51:11.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:11.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:51:12.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:13.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:13.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:13.012 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:51:13.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:51:13.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:13.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:51:13.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:13.258 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:13.248+0000 7f4e4f0cc8c0 -1 Falling back to public interface 2026-02-17T11:51:14.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:14.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:14.162 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:51:14.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:51:14.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:14.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:51:14.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:15.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:15.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:15.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:51:15.311 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:51:15.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:15.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:51:15.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:16.214 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:16.204+0000 7f4e4f0cc8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:51:16.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:16.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:16.464 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:51:16.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:51:16.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:16.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:51:16.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:17.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:17.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:17.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:51:17.620 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:51:17.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:17.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:51:17.770 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3892745070,v1:127.0.0.1:6803/3892745070] [v2:127.0.0.1:6804/3892745070,v1:127.0.0.1:6805/3892745070] exists,up 66c875c2-c236-40bb-9241-94fcfe167999 2026-02-17T11:51:17.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:51:17.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:51:17.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:51:17.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:100: TEST_corrupt_and_repair_replicated: run_osd td/osd-scrub-repair 1 2026-02-17T11:51:17.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:51:17.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:51:17.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T11:51:17.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:51:17.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:51:17.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:51:17.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:51:17.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:51:17.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:51:17.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:51:17.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:51:17.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:51:17.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:51:17.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:51:17.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:51:17.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:51:17.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:51:17.775 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:51:17.776 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:51:17.776 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:51:17.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:51:17.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:51:17.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:51:17.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:51:17.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:51:17.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:51:17.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:51:17.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:51:17.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:51:17.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:51:17.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:51:17.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:51:17.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:51:17.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:51:17.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:51:17.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=69ff3b7b-d527-4ca1-acfb-c6b6b45acb8c 2026-02-17T11:51:17.778 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 69ff3b7b-d527-4ca1-acfb-c6b6b45acb8c 2026-02-17T11:51:17.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 69ff3b7b-d527-4ca1-acfb-c6b6b45acb8c' 2026-02-17T11:51:17.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:51:17.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA1VpRpM+50LhAAtoqfmyvqum7yZhmimsaIJg== 2026-02-17T11:51:17.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA1VpRpM+50LhAAtoqfmyvqum7yZhmimsaIJg=="}' 2026-02-17T11:51:17.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 69ff3b7b-d527-4ca1-acfb-c6b6b45acb8c -i td/osd-scrub-repair/1/new.json 2026-02-17T11:51:17.940 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:51:17.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T11:51:17.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA1VpRpM+50LhAAtoqfmyvqum7yZhmimsaIJg== --osd-uuid 69ff3b7b-d527-4ca1-acfb-c6b6b45acb8c 2026-02-17T11:51:17.966 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:17.956+0000 7f4ecb8548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:17.967 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:17.956+0000 7f4ecb8548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:17.968 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:17.956+0000 7f4ecb8548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:17.969 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:17.956+0000 7f4ecb8548c0 -1 bdev(0x56013b3b9c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:51:17.969 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:17.956+0000 7f4ecb8548c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T11:51:20.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T11:51:20.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:51:20.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T11:51:20.255 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:51:20.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:51:20.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T11:51:20.447 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:51:20.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:51:20.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:51:20.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:51:20.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:51:20.470 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:20.460+0000 7fb6581e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:20.472 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:20.460+0000 7fb6581e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:20.473 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:20.464+0000 7fb6581e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:20.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T11:51:20.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:51:20.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:51:20.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:51:20.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:51:20.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:20.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:51:20.604 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:51:20.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:20.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:51:20.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:21.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:21.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:21.755 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:51:21.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:51:21.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:21.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:51:21.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:21.958 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:21.948+0000 7fb6581e08c0 -1 Falling back to public interface 2026-02-17T11:51:22.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:22.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:22.906 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:51:22.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:51:22.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:22.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:51:23.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:24.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:24.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:24.064 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:51:24.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:51:24.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:24.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:51:24.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:24.978 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:24.968+0000 7fb6581e08c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:51:25.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:25.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:25.216 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:51:25.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:51:25.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:25.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:51:25.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:26.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:26.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:26.370 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:51:26.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:51:26.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:26.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:51:26.524 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3793596066,v1:127.0.0.1:6811/3793596066] [v2:127.0.0.1:6812/3793596066,v1:127.0.0.1:6813/3793596066] exists,up 69ff3b7b-d527-4ca1-acfb-c6b6b45acb8c 2026-02-17T11:51:26.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:51:26.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:51:26.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:51:26.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:101: TEST_corrupt_and_repair_replicated: create_rbd_pool 2026-02-17T11:51:26.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T11:51:26.666 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' does not exist 2026-02-17T11:51:26.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T11:51:26.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T11:51:26.891 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T11:51:26.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:51:27.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T11:51:28.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:102: TEST_corrupt_and_repair_replicated: wait_for_clean 2026-02-17T11:51:28.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:51:28.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:51:28.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:51:28.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:51:28.187 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:51:28.187 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:51:28.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:51:28.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:51:28.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:51:28.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:51:28.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:51:28.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:51:28.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:51:28.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:51:28.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:51:28.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:51:28.448 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:51:28.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:51:28.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:28.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:51:28.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T11:51:28.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T11:51:28.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T11:51:28.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:28.520 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:51:28.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705666 2026-02-17T11:51:28.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705666 2026-02-17T11:51:28.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-38654705666' 2026-02-17T11:51:28.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:28.591 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T11:51:28.592 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:28.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:51:28.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T11:51:28.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:28.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T11:51:28.595 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836484 2026-02-17T11:51:28.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T11:51:28.596 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:51:28.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T11:51:28.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:51:29.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:51:29.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:51:29.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T11:51:29.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:29.908 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705666 2026-02-17T11:51:29.908 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:29.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:51:29.909 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705666 2026-02-17T11:51:29.910 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:29.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705666 2026-02-17T11:51:29.911 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705666 2026-02-17T11:51:29.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705666' 2026-02-17T11:51:29.912 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:51:30.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705666 -lt 38654705666 2026-02-17T11:51:30.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:51:30.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:51:30.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:51:30.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T11:51:30.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:51:30.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:51:30.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:51:30.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:51:30.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:51:30.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:51:30.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:51:30.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T11:51:30.396 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:51:30.397 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:51:30.397 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:51:30.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T11:51:30.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:51:30.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:51:30.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:104: TEST_corrupt_and_repair_replicated: add_something td/osd-scrub-repair rbd 2026-02-17T11:51:30.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:51:30.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=rbd 2026-02-17T11:51:30.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T11:51:30.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:51:30.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:51:30.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:51:30.772 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:51:30.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:51:30.977 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:51:30.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:51:30.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:51:30.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool rbd put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T11:51:31.014 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:105: TEST_corrupt_and_repair_replicated: get_not_primary rbd SOMETHING 2026-02-17T11:51:31.014 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=rbd 2026-02-17T11:51:31.014 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T11:51:31.015 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary rbd SOMETHING 2026-02-17T11:51:31.015 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=rbd 2026-02-17T11:51:31.015 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:51:31.015 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map rbd SOMETHING 2026-02-17T11:51:31.015 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:51:31.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T11:51:31.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map rbd SOMETHING 2026-02-17T11:51:31.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T11:51:31.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:105: TEST_corrupt_and_repair_replicated: corrupt_and_repair_one td/osd-scrub-repair rbd 0 2026-02-17T11:51:31.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T11:51:31.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=rbd 2026-02-17T11:51:31.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=0 2026-02-17T11:51:31.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T11:51:31.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:51:31.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:51:31.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:51:31.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:51:31.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T11:51:31.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:51:31.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:51:31.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:51:31.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:51:31.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:51:31.329 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:51:31.329 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:51:31.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:51:31.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:51:31.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:51:32.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:51:32.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T11:51:32.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:51:32.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:51:32.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:51:32.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:51:32.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:51:32.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-17T11:51:33.279 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-17T11:51:33.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:51:33.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:51:33.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:51:33.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:51:33.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:51:33.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:51:33.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:51:33.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:51:33.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:51:33.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:51:33.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:51:33.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:51:33.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:51:33.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:51:33.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:51:33.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:51:33.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:51:33.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:51:33.827 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:51:33.827 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:51:33.827 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:51:33.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:51:33.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:51:33.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:51:33.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:51:33.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:51:33.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:51:33.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:51:33.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:51:33.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:51:33.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:51:33.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:51:33.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:51:33.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:51:33.830 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:51:33.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:51:33.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:51:33.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:51:33.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:51:33.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:51:33.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:51:33.849 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:33.840+0000 7f911e9dc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:33.851 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:33.840+0000 7f911e9dc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:33.852 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:33.840+0000 7f911e9dc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:33.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:51:33.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:51:33.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:51:33.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:51:33.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:51:33.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:33.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:51:33.987 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:51:33.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:33.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:51:34.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:35.127 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:35.116+0000 7f911e9dc8c0 -1 Falling back to public interface 2026-02-17T11:51:35.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:35.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:35.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:51:35.139 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:51:35.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:35.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:51:35.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:36.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:36.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:36.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:51:36.289 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:51:36.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:36.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:51:36.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:37.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:37.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:37.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:51:37.440 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:51:37.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:37.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:51:37.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:38.096 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:38.088+0000 7f911e9dc8c0 -1 osd.0 18 log_to_monitors true 2026-02-17T11:51:38.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:38.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:38.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:51:38.590 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:51:38.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:38.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:51:38.738 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 23 up_thru 23 down_at 19 last_clean_interval [5,18) [v2:127.0.0.1:6802/1475697348,v1:127.0.0.1:6803/1475697348] [v2:127.0.0.1:6804/1475697348,v1:127.0.0.1:6805/1475697348] exists,up 66c875c2-c236-40bb-9241-94fcfe167999 2026-02-17T11:51:38.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:51:38.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:51:38.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:51:38.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:51:38.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:51:38.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:51:38.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:51:38.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:51:38.739 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:51:38.739 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:51:38.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:51:38.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:51:38.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:51:38.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:51:38.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:51:38.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:51:38.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:51:38.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:51:38.851 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:51:39.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:51:39.008 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:51:39.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:51:39.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:39.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:51:39.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247810 2026-02-17T11:51:39.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247810 2026-02-17T11:51:39.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810' 2026-02-17T11:51:39.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:39.078 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:51:39.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-17T11:51:39.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-17T11:51:39.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810 1-38654705669' 2026-02-17T11:51:39.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:39.151 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247810 2026-02-17T11:51:39.151 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:39.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:51:39.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247810 2026-02-17T11:51:39.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:39.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247810 2026-02-17T11:51:39.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247810' 2026-02-17T11:51:39.155 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 98784247810 2026-02-17T11:51:39.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:51:39.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 98784247810 2026-02-17T11:51:39.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:51:40.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:51:40.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:51:40.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247810 -lt 98784247810 2026-02-17T11:51:40.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:40.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-17T11:51:40.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:40.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:51:40.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-17T11:51:40.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:40.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-17T11:51:40.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-17T11:51:40.473 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705669 2026-02-17T11:51:40.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:51:40.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-17T11:51:40.627 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:51:40.627 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:51:40.627 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:51:40.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T11:51:40.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:51:40.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:51:40.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:51:40.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:51:40.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:51:40.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:51:40.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:51:40.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T11:51:40.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:51:40.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:51:40.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:51:41.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T11:51:41.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:51:41.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:51:41.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg rbd SOMETHING 2026-02-17T11:51:41.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=rbd 2026-02-17T11:51:41.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:51:41.156 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map rbd SOMETHING 2026-02-17T11:51:41.156 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:51:41.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=1.3 2026-02-17T11:51:41.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 1.3 2026-02-17T11:51:41.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.3 2026-02-17T11:51:41.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.3 2026-02-17T11:51:41.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:51:41.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:51:41.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:51:41.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:51:41.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:51:26.883469+0000 2026-02-17T11:51:41.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.3 2026-02-17T11:51:41.608 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 1.3 on osd.1 to repair 2026-02-17T11:51:41.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.3 2026-02-17T11:51:26.883469+0000 2026-02-17T11:51:41.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.3 2026-02-17T11:51:41.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:51:26.883469+0000 2026-02-17T11:51:41.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:51:41.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:51:41.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:51:41.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T11:51:41.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:51:41.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:51:41.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:51:41.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:51:41.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:51:26.883469+0000 '>' 2026-02-17T11:51:26.883469+0000 2026-02-17T11:51:41.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:51:42.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:51:42.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:51:42.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T11:51:42.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:51:42.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:51:42.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:51:42.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:51:42.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:51:26.883469+0000 '>' 2026-02-17T11:51:26.883469+0000 2026-02-17T11:51:42.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:51:43.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:51:43.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:51:43.932 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T11:51:43.932 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:51:43.932 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:51:43.932 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:51:43.932 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:51:44.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:51:26.883469+0000 '>' 2026-02-17T11:51:26.883469+0000 2026-02-17T11:51:44.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:51:45.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:51:45.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:51:45.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T11:51:45.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:51:45.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:51:45.087 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:51:45.087 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:51:45.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:51:26.883469+0000 '>' 2026-02-17T11:51:26.883469+0000 2026-02-17T11:51:45.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:51:46.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:51:46.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:51:46.243 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T11:51:46.243 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:51:46.243 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:51:46.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:51:46.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:51:46.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:51:26.883469+0000 '>' 2026-02-17T11:51:26.883469+0000 2026-02-17T11:51:46.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:51:47.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:51:47.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:51:47.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T11:51:47.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:51:47.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:51:47.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:51:47.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:51:47.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:51:41.877008+0000 '>' 2026-02-17T11:51:26.883469+0000 2026-02-17T11:51:47.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:51:47.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:51:47.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:51:47.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:51:47.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:51:47.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:51:47.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:51:47.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:51:47.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:51:47.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:51:47.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:51:47.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:51:47.555 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:51:47.555 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:51:47.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:51:47.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:51:47.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:51:48.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:51:48.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:51:48.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:51:48.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:51:48.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:51:48.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:51:48.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:51:48.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T11:51:49.187 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::scrub_1.3:head#, (61) No data available 2026-02-17T11:51:49.187 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:51:49.188 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:51:49.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:51:49.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:51:49.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:51:49.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:51:49.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:51:49.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:51:49.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:51:49.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:51:49.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:51:49.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:51:49.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:51:49.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:51:49.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:51:49.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:51:49.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:51:49.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:51:49.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:51:49.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:51:49.725 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:51:49.725 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:51:49.725 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:51:49.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:51:49.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:51:49.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:51:49.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:51:49.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:51:49.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:51:49.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:51:49.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:51:49.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:51:49.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:51:49.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:51:49.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:51:49.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:51:49.729 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:51:49.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:51:49.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:51:49.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:51:49.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:51:49.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:51:49.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:51:49.744 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:49.732+0000 7f6837f018c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:49.745 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:49.736+0000 7f6837f018c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:49.746 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:49.736+0000 7f6837f018c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:49.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:51:49.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:51:49.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:51:49.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:51:49.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:51:49.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:49.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:51:49.883 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:51:49.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:49.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:51:50.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:51.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:51.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:51.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:51:51.036 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:51:51.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:51.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:51:51.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:51.254 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:51.244+0000 7f6837f018c0 -1 Falling back to public interface 2026-02-17T11:51:52.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:52.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:52.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:51:52.190 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:51:52.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:52.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:51:52.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:53.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:53.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:53.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:51:53.340 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:51:53.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:53.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:51:53.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:51:54.264 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:54.252+0000 7f6837f018c0 -1 osd.1 26 log_to_monitors true 2026-02-17T11:51:54.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:51:54.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:54.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:51:54.492 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:51:54.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:54.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:51:54.641 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 30 up_thru 30 down_at 27 last_clean_interval [9,26) [v2:127.0.0.1:6810/676028121,v1:127.0.0.1:6811/676028121] [v2:127.0.0.1:6812/676028121,v1:127.0.0.1:6813/676028121] exists,up 69ff3b7b-d527-4ca1-acfb-c6b6b45acb8c 2026-02-17T11:51:54.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:51:54.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:51:54.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:51:54.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:51:54.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:51:54.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:51:54.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:51:54.641 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:51:54.642 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:51:54.642 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:51:54.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:51:54.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:51:54.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:51:54.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:51:54.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:51:54.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:51:54.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:51:54.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:51:54.753 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:51:54.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:51:54.903 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:51:54.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:51:54.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:54.903 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:51:54.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247814 2026-02-17T11:51:54.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247814 2026-02-17T11:51:54.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247814' 2026-02-17T11:51:54.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:51:54.976 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:51:55.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018882 2026-02-17T11:51:55.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018882 2026-02-17T11:51:55.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247814 1-128849018882' 2026-02-17T11:51:55.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:55.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247814 2026-02-17T11:51:55.049 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:55.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:51:55.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247814 2026-02-17T11:51:55.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:55.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247814 2026-02-17T11:51:55.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247814' 2026-02-17T11:51:55.053 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 98784247814 2026-02-17T11:51:55.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:51:55.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247813 -lt 98784247814 2026-02-17T11:51:55.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:51:56.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:51:56.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:51:56.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247814 -lt 98784247814 2026-02-17T11:51:56.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:51:56.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-128849018882 2026-02-17T11:51:56.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:51:56.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:51:56.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-128849018882 2026-02-17T11:51:56.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:51:56.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018882 2026-02-17T11:51:56.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 128849018882' 2026-02-17T11:51:56.364 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 128849018882 2026-02-17T11:51:56.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:51:56.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018882 -lt 128849018882 2026-02-17T11:51:56.519 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:51:56.520 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:51:56.520 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:51:56.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T11:51:56.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:51:56.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:51:56.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:51:56.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:51:56.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:51:56.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:51:56.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:51:56.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T11:51:56.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:51:56.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:51:56.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:51:57.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T11:51:57.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:51:57.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:51:57.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool rbd get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:51:57.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:51:57.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:107: TEST_corrupt_and_repair_replicated: get_primary rbd SOMETHING 2026-02-17T11:51:57.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=rbd 2026-02-17T11:51:57.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T11:51:57.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map rbd SOMETHING 2026-02-17T11:51:57.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:51:57.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:107: TEST_corrupt_and_repair_replicated: corrupt_and_repair_one td/osd-scrub-repair rbd 1 2026-02-17T11:51:57.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:226: corrupt_and_repair_one: local dir=td/osd-scrub-repair 2026-02-17T11:51:57.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:227: corrupt_and_repair_one: local poolname=rbd 2026-02-17T11:51:57.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:228: corrupt_and_repair_one: local osd=1 2026-02-17T11:51:57.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:233: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:51:57.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:51:57.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:51:57.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:51:57.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:51:57.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:51:57.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:51:57.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:51:57.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:51:57.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:51:57.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:51:57.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:51:57.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:51:57.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:51:57.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:51:57.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:51:58.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:51:58.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING remove 2026-02-17T11:51:58.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:51:58.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:51:58.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:51:58.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:51:58.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:51:58.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING remove 2026-02-17T11:51:59.185 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:eb822e21:::SOMETHING:head# 2026-02-17T11:51:59.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:51:59.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:51:59.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:51:59.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:51:59.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:51:59.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:51:59.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:51:59.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:51:59.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:51:59.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:51:59.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:51:59.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:51:59.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:51:59.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:51:59.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:51:59.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:51:59.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:51:59.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:51:59.725 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:51:59.725 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:51:59.725 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:51:59.726 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:51:59.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:51:59.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:51:59.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:51:59.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:51:59.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:51:59.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:51:59.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:51:59.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:51:59.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:51:59.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:51:59.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:51:59.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:51:59.727 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:51:59.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:51:59.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:51:59.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:51:59.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:51:59.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:51:59.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:51:59.748 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:59.740+0000 7f12ea0658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:59.750 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:59.740+0000 7f12ea0658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:59.751 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:51:59.740+0000 7f12ea0658c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:51:59.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:51:59.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:51:59.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:51:59.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:51:59.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:51:59.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:51:59.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:51:59.883 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:51:59.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:51:59.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:52:00.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:01.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:01.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:01.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:52:01.035 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:52:01.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:01.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:52:01.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:01.242 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:01.232+0000 7f12ea0658c0 -1 Falling back to public interface 2026-02-17T11:52:02.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:02.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:02.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:52:02.186 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:52:02.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:02.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:52:02.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:03.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:03.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:03.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:52:03.337 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:52:03.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:03.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:52:03.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:04.267 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:04.256+0000 7f12ea0658c0 -1 osd.1 32 log_to_monitors true 2026-02-17T11:52:04.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:04.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:04.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:52:04.489 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:52:04.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:04.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:52:04.645 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 36 up_thru 36 down_at 33 last_clean_interval [30,32) [v2:127.0.0.1:6810/123964658,v1:127.0.0.1:6811/123964658] [v2:127.0.0.1:6812/123964658,v1:127.0.0.1:6813/123964658] exists,up 69ff3b7b-d527-4ca1-acfb-c6b6b45acb8c 2026-02-17T11:52:04.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:52:04.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:52:04.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:52:04.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:52:04.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:52:04.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:52:04.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:52:04.646 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:52:04.647 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:52:04.647 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:52:04.647 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:52:04.647 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:52:04.647 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:52:04.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:52:04.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:52:04.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:52:04.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:52:04.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:52:04.757 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:52:04.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:52:04.915 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:52:04.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:52:04.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:52:04.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:52:04.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247817 2026-02-17T11:52:04.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247817 2026-02-17T11:52:04.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247817' 2026-02-17T11:52:04.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:52:04.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:52:05.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822658 2026-02-17T11:52:05.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822658 2026-02-17T11:52:05.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247817 1-154618822658' 2026-02-17T11:52:05.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:52:05.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247817 2026-02-17T11:52:05.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:52:05.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:52:05.065 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247817 2026-02-17T11:52:05.065 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:52:05.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247817 2026-02-17T11:52:05.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247817' 2026-02-17T11:52:05.067 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 98784247817 2026-02-17T11:52:05.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:52:05.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247816 -lt 98784247817 2026-02-17T11:52:05.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:52:06.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:52:06.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:52:06.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247817 -lt 98784247817 2026-02-17T11:52:06.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:52:06.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-154618822658 2026-02-17T11:52:06.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:52:06.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:52:06.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-154618822658 2026-02-17T11:52:06.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:52:06.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822658 2026-02-17T11:52:06.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 154618822658' 2026-02-17T11:52:06.392 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 154618822658 2026-02-17T11:52:06.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:52:06.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822658 -lt 154618822658 2026-02-17T11:52:06.552 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:52:06.552 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:52:06.552 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:52:06.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T11:52:06.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:52:06.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:52:06.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:52:06.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:52:06.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:52:06.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:52:06.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:52:06.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T11:52:06.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:52:06.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:52:06.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:52:07.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T11:52:07.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:52:07.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:52:07.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: get_pg rbd SOMETHING 2026-02-17T11:52:07.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=rbd 2026-02-17T11:52:07.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T11:52:07.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map rbd SOMETHING 2026-02-17T11:52:07.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:52:07.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:237: corrupt_and_repair_one: local pg=1.3 2026-02-17T11:52:07.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:238: corrupt_and_repair_one: repair 1.3 2026-02-17T11:52:07.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.3 2026-02-17T11:52:07.218 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.3 2026-02-17T11:52:07.218 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:52:07.218 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:52:07.218 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:52:07.218 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:52:07.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T11:51:41.877008+0000 2026-02-17T11:52:07.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.3 2026-02-17T11:52:07.514 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 1.3 on osd.1 to repair 2026-02-17T11:52:07.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.3 2026-02-17T11:51:41.877008+0000 2026-02-17T11:52:07.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.3 2026-02-17T11:52:07.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:51:41.877008+0000 2026-02-17T11:52:07.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:52:07.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:52:07.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:52:07.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T11:52:07.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:52:07.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:52:07.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:52:07.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:52:07.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:51:41.877008+0000 '>' 2026-02-17T11:51:41.877008+0000 2026-02-17T11:52:07.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:52:08.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:52:08.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:52:08.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T11:52:08.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:52:08.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:52:08.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:52:08.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:52:08.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:51:41.877008+0000 '>' 2026-02-17T11:51:41.877008+0000 2026-02-17T11:52:08.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:52:09.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:52:09.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:52:09.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T11:52:09.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:52:09.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:52:09.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:52:09.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:52:09.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:51:41.877008+0000 '>' 2026-02-17T11:51:41.877008+0000 2026-02-17T11:52:09.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:52:10.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:52:10.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:52:10.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T11:52:10.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:52:10.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:52:10.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:52:10.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:52:11.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:51:41.877008+0000 '>' 2026-02-17T11:51:41.877008+0000 2026-02-17T11:52:11.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:52:12.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:52:12.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:52:12.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T11:52:12.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T11:52:12.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:52:12.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:52:12.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T11:52:12.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:52:08.197198+0000 '>' 2026-02-17T11:51:41.877008+0000 2026-02-17T11:52:12.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:52:12.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:242: corrupt_and_repair_one: objectstore_tool td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:52:12.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:52:12.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:52:12.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:52:12.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:52:12.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:52:12.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:52:12.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:52:12.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:52:12.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:52:12.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:52:12.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:52:12.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:52:12.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:52:12.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:52:12.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:52:13.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:52:13.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 SOMETHING list-attrs 2026-02-17T11:52:13.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:52:13.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:52:13.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:52:13.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:52:13.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:52:13.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 SOMETHING list-attrs 2026-02-17T11:52:13.901 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.3_head,#-3:c0000000:::scrub_1.3:head#, (61) No data available 2026-02-17T11:52:13.901 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T11:52:13.901 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T11:52:14.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:52:14.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:52:14.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:52:14.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:52:14.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:52:14.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:52:14.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:52:14.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:52:14.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:52:14.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:52:14.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:52:14.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:52:14.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:52:14.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:52:14.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:52:14.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:52:14.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:52:14.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:52:14.440 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:52:14.440 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:52:14.440 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:52:14.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:52:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:52:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:52:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:52:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:52:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:52:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:52:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:52:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:52:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:52:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:52:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:52:14.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:52:14.441 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:52:14.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:52:14.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:52:14.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:52:14.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:52:14.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:52:14.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:52:14.463 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:14.451+0000 7f163eb528c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:14.464 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:14.455+0000 7f163eb528c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:14.465 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:14.455+0000 7f163eb528c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:14.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:52:14.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:52:14.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:52:14.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:52:14.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:52:14.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:14.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:52:14.600 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:52:14.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:14.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:52:14.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:15.722 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:15.711+0000 7f163eb528c0 -1 Falling back to public interface 2026-02-17T11:52:15.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:15.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:15.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:52:15.757 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:52:15.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:15.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:52:15.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:16.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:16.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:16.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:52:16.911 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:52:16.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:16.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:52:17.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:18.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:18.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:18.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:52:18.072 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:52:18.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:18.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:52:18.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:18.700 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:18.691+0000 7f163eb528c0 -1 osd.1 37 log_to_monitors true 2026-02-17T11:52:19.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:19.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:19.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:52:19.228 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:52:19.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:19.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:52:19.384 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 41 up_thru 41 down_at 38 last_clean_interval [36,37) [v2:127.0.0.1:6810/1453037952,v1:127.0.0.1:6811/1453037952] [v2:127.0.0.1:6812/1453037952,v1:127.0.0.1:6813/1453037952] exists,up 69ff3b7b-d527-4ca1-acfb-c6b6b45acb8c 2026-02-17T11:52:19.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:52:19.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:52:19.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:52:19.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:52:19.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:52:19.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:52:19.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:52:19.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:52:19.385 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:52:19.386 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:52:19.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:52:19.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:52:19.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:52:19.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:52:19.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:52:19.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:52:19.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:52:19.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:52:19.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:52:19.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:52:19.650 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:52:19.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:52:19.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:52:19.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:52:19.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247821 2026-02-17T11:52:19.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247821 2026-02-17T11:52:19.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247821' 2026-02-17T11:52:19.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:52:19.726 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:52:19.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659138 2026-02-17T11:52:19.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659138 2026-02-17T11:52:19.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247821 1-176093659138' 2026-02-17T11:52:19.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:52:19.800 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247821 2026-02-17T11:52:19.800 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:52:19.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:52:19.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247821 2026-02-17T11:52:19.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:52:19.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247821 2026-02-17T11:52:19.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247821' 2026-02-17T11:52:19.804 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 98784247821 2026-02-17T11:52:19.804 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:52:19.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247820 -lt 98784247821 2026-02-17T11:52:19.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:52:20.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:52:20.964 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:52:21.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247820 -lt 98784247821 2026-02-17T11:52:21.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:52:22.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:52:22.128 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:52:22.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247821 -lt 98784247821 2026-02-17T11:52:22.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:52:22.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-176093659138 2026-02-17T11:52:22.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:52:22.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:52:22.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-176093659138 2026-02-17T11:52:22.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:52:22.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659138 2026-02-17T11:52:22.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 176093659138' 2026-02-17T11:52:22.285 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 176093659138 2026-02-17T11:52:22.285 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:52:22.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659138 -lt 176093659138 2026-02-17T11:52:22.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:52:22.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:52:22.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:52:22.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T11:52:22.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:52:22.617 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:52:22.617 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:52:22.617 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:52:22.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:52:22.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:52:22.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:52:22.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T11:52:22.764 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:52:22.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:52:22.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:52:22.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T11:52:22.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:52:22.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:52:22.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:243: corrupt_and_repair_one: rados --pool rbd get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T11:52:22.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:244: corrupt_and_repair_one: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T11:52:22.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T11:52:22.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:52:22.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:52:22.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:52:22.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:52:22.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:52:22.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:52:22.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:52:22.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:52:23.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:52:23.074 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:52:23.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:52:23.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:52:23.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:52:23.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:52:23.077 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:52:23.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:52:23.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:52:23.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:52:23.079 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:52:23.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:52:23.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:52:23.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:52:23.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:52:23.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:52:23.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:52:23.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:52:23.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:52:23.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:52:23.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T11:52:23.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T11:52:23.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T11:52:23.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T11:52:23.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:52:23.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:52:23.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:52:23.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:52:23.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:52:23.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:52:23.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:52:23.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:52:23.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:52:23.099 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:52:23.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:52:23.101 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:52:23.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:52:23.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:52:23.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:52:23.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:52:23.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:52:23.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:52:23.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:52:23.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:52:23.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:52:23.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:52:23.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:52:23.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:52:23.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:52:23.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:52:23.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:52:23.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:52:23.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T11:52:23.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:52:23.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:52:23.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:52:23.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T11:52:23.113 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:52:23.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T11:52:23.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:52:23.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T11:52:23.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_scrub_erasure_appends td/osd-scrub-repair 2026-02-17T11:52:23.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5746: TEST_corrupt_scrub_erasure_appends: corrupt_scrub_erasure td/osd-scrub-repair false 2026-02-17T11:52:23.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3596: corrupt_scrub_erasure: local dir=td/osd-scrub-repair 2026-02-17T11:52:23.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3597: corrupt_scrub_erasure: local allow_overwrites=false 2026-02-17T11:52:23.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3598: corrupt_scrub_erasure: local poolname=ecpool 2026-02-17T11:52:23.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3599: corrupt_scrub_erasure: local total_objs=7 2026-02-17T11:52:23.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3601: corrupt_scrub_erasure: run_mon td/osd-scrub-repair a 2026-02-17T11:52:23.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T11:52:23.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T11:52:23.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T11:52:23.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T11:52:23.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T11:52:23.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T11:52:23.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T11:52:23.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:52:23.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:52:23.136 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:52:23.136 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:52:23.136 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:52:23.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:52:23.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T11:52:23.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T11:52:23.165 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T11:52:23.165 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:52:23.165 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:52:23.165 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T11:52:23.165 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T11:52:23.165 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:52:23.165 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:52:23.165 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:52:23.166 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:52:23.166 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:52:23.167 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:52:23.167 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:52:23.167 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:52:23.167 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T11:52:23.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T11:52:23.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:52:23.224 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:52:23.224 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T11:52:23.224 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T11:52:23.224 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:52:23.224 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:52:23.224 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:52:23.225 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:52:23.225 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:52:23.225 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:52:23.225 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:52:23.225 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:52:23.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T11:52:23.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3602: corrupt_scrub_erasure: run_mgr td/osd-scrub-repair x 2026-02-17T11:52:23.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T11:52:23.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T11:52:23.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T11:52:23.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T11:52:23.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T11:52:23.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T11:52:23.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T11:52:23.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:52:23.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:52:23.376 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:52:23.376 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:52:23.376 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:52:23.376 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:52:23.376 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:52:23.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:52:23.396 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3603: corrupt_scrub_erasure: seq 0 2 2026-02-17T11:52:23.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3603: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-17T11:52:23.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3604: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 0 2026-02-17T11:52:23.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:52:23.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:52:23.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T11:52:23.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:52:23.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:52:23.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:52:23.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:52:23.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:52:23.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:52:23.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:52:23.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:52:23.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:52:23.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:52:23.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:52:23.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:52:23.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:52:23.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:52:23.402 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:52:23.402 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:52:23.402 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:52:23.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:52:23.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:52:23.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:52:23.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:52:23.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:52:23.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:52:23.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:52:23.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:52:23.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:52:23.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:52:23.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:52:23.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:52:23.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:52:23.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:52:23.404 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:52:23.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b7951352-7c09-4b75-abda-71432d2b706a 2026-02-17T11:52:23.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 b7951352-7c09-4b75-abda-71432d2b706a' 2026-02-17T11:52:23.405 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 b7951352-7c09-4b75-abda-71432d2b706a 2026-02-17T11:52:23.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:52:23.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB3VpRpw2oqGBAAQYPIX4Cw+yDyNf5lHGftUA== 2026-02-17T11:52:23.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB3VpRpw2oqGBAAQYPIX4Cw+yDyNf5lHGftUA=="}' 2026-02-17T11:52:23.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b7951352-7c09-4b75-abda-71432d2b706a -i td/osd-scrub-repair/0/new.json 2026-02-17T11:52:23.501 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:52:23.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T11:52:23.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB3VpRpw2oqGBAAQYPIX4Cw+yDyNf5lHGftUA== --osd-uuid b7951352-7c09-4b75-abda-71432d2b706a 2026-02-17T11:52:23.526 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:23.515+0000 7fde7bb248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:23.527 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:23.515+0000 7fde7bb248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:23.528 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:23.515+0000 7fde7bb248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:23.528 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:23.519+0000 7fde7bb248c0 -1 bdev(0x55aa30b6fc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:52:23.529 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:23.519+0000 7fde7bb248c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T11:52:25.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T11:52:25.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:52:25.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T11:52:25.837 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:52:25.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:52:26.026 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:52:26.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T11:52:26.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:52:26.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:52:26.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:52:26.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:52:26.050 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:26.039+0000 7f6aebf568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:26.052 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:26.039+0000 7f6aebf568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:26.053 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:26.043+0000 7f6aebf568c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:26.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T11:52:26.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:52:26.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:52:26.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:52:26.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:52:26.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:26.179 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:52:26.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:52:26.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:26.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:52:26.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:27.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:27.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:27.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:52:27.336 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:52:27.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:27.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:52:27.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:27.542 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:27.531+0000 7f6aebf568c0 -1 Falling back to public interface 2026-02-17T11:52:28.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:28.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:28.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:52:28.488 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:52:28.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:28.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:52:28.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:29.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:29.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:29.646 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:52:29.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:52:29.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:29.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:52:29.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:30.503 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:30.491+0000 7f6aebf568c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:52:30.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:30.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:30.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:52:30.801 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:52:30.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:30.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:52:30.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:31.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:31.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:31.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:52:31.959 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:52:31.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:31.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:52:32.114 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3536177968,v1:127.0.0.1:6803/3536177968] [v2:127.0.0.1:6804/3536177968,v1:127.0.0.1:6805/3536177968] exists,up b7951352-7c09-4b75-abda-71432d2b706a 2026-02-17T11:52:32.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:52:32.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:52:32.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:52:32.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3603: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-17T11:52:32.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3604: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 1 2026-02-17T11:52:32.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:52:32.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:52:32.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T11:52:32.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:52:32.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:52:32.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:52:32.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:52:32.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:52:32.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:52:32.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:52:32.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:52:32.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:52:32.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:52:32.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:52:32.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:52:32.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:52:32.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:52:32.120 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:52:32.120 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:52:32.120 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:52:32.120 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:52:32.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:52:32.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:52:32.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:52:32.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:52:32.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:52:32.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:52:32.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:52:32.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:52:32.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:52:32.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:52:32.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:52:32.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:52:32.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:52:32.121 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:52:32.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=5280bdd0-cca7-4c9e-8ce1-9ce91b6db0a3 2026-02-17T11:52:32.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 5280bdd0-cca7-4c9e-8ce1-9ce91b6db0a3' 2026-02-17T11:52:32.122 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 5280bdd0-cca7-4c9e-8ce1-9ce91b6db0a3 2026-02-17T11:52:32.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:52:32.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCAVpRpUyVkBxAAm6Y6ZkRjYKLdtc4Xf/0awA== 2026-02-17T11:52:32.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCAVpRpUyVkBxAAm6Y6ZkRjYKLdtc4Xf/0awA=="}' 2026-02-17T11:52:32.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 5280bdd0-cca7-4c9e-8ce1-9ce91b6db0a3 -i td/osd-scrub-repair/1/new.json 2026-02-17T11:52:32.291 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:52:32.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T11:52:32.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCAVpRpUyVkBxAAm6Y6ZkRjYKLdtc4Xf/0awA== --osd-uuid 5280bdd0-cca7-4c9e-8ce1-9ce91b6db0a3 2026-02-17T11:52:32.325 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:32.315+0000 7f25873bd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:32.327 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:32.315+0000 7f25873bd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:32.328 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:32.315+0000 7f25873bd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:32.328 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:32.319+0000 7f25873bd8c0 -1 bdev(0x55a3b58ffc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:52:32.328 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:32.319+0000 7f25873bd8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T11:52:34.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T11:52:34.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:52:34.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T11:52:34.603 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:52:34.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:52:34.804 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:52:34.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T11:52:34.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:52:34.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:52:34.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:52:34.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:52:34.827 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:34.815+0000 7fda8bb4b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:34.829 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:34.819+0000 7fda8bb4b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:34.830 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:34.819+0000 7fda8bb4b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:34.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T11:52:34.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:52:34.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:52:34.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:52:34.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:52:34.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:34.955 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:52:34.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:52:34.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:34.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:52:35.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:36.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:36.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:36.111 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:52:36.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:52:36.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:36.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:52:36.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:36.310 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:36.299+0000 7fda8bb4b8c0 -1 Falling back to public interface 2026-02-17T11:52:37.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:37.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:37.264 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:52:37.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:52:37.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:37.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:52:37.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:38.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:38.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:38.424 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:52:38.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:52:38.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:38.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:52:38.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:39.336 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:39.323+0000 7fda8bb4b8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:52:39.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:39.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:39.581 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:52:39.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:52:39.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:39.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:52:39.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:40.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:40.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:40.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:52:40.735 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:52:40.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:40.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:52:40.889 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/167209839,v1:127.0.0.1:6811/167209839] [v2:127.0.0.1:6812/167209839,v1:127.0.0.1:6813/167209839] exists,up 5280bdd0-cca7-4c9e-8ce1-9ce91b6db0a3 2026-02-17T11:52:40.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:52:40.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:52:40.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:52:40.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3603: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-17T11:52:40.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3604: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 2 2026-02-17T11:52:40.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:52:40.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:52:40.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T11:52:40.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:52:40.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:52:40.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:52:40.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:52:40.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:52:40.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:52:40.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:52:40.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:52:40.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:52:40.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:52:40.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:52:40.895 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:52:40.895 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:52:40.895 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:52:40.895 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:52:40.895 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:52:40.895 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:52:40.895 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:52:40.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:52:40.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:52:40.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:52:40.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:52:40.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:52:40.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:52:40.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:52:40.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:52:40.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:52:40.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:52:40.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:52:40.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:52:40.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:52:40.897 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:52:40.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=f7cd5d50-45a2-4c55-a175-2991537e538f 2026-02-17T11:52:40.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 f7cd5d50-45a2-4c55-a175-2991537e538f' 2026-02-17T11:52:40.898 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 f7cd5d50-45a2-4c55-a175-2991537e538f 2026-02-17T11:52:40.898 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:52:40.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCIVpRp9x+dNRAAwSpmK7uOB8qYM1mhIcOdfw== 2026-02-17T11:52:40.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCIVpRp9x+dNRAAwSpmK7uOB8qYM1mhIcOdfw=="}' 2026-02-17T11:52:40.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new f7cd5d50-45a2-4c55-a175-2991537e538f -i td/osd-scrub-repair/2/new.json 2026-02-17T11:52:41.067 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:52:41.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T11:52:41.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCIVpRp9x+dNRAAwSpmK7uOB8qYM1mhIcOdfw== --osd-uuid f7cd5d50-45a2-4c55-a175-2991537e538f 2026-02-17T11:52:41.092 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:41.083+0000 7fea73b1e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:41.093 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:41.083+0000 7fea73b1e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:41.095 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:41.083+0000 7fea73b1e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:41.095 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:41.083+0000 7fea73b1e8c0 -1 bdev(0x558ca4e6bc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T11:52:41.095 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:41.083+0000 7fea73b1e8c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T11:52:43.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T11:52:43.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:52:43.391 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T11:52:43.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T11:52:43.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:52:43.590 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T11:52:43.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T11:52:43.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:52:43.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:52:43.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:52:43.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:52:43.613 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:43.603+0000 7f02b5c5d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:43.615 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:43.603+0000 7f02b5c5d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:43.616 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:43.603+0000 7f02b5c5d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:52:43.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T11:52:43.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:52:43.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:52:43.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:52:43.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:52:43.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:43.741 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:52:43.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:52:43.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:43.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:52:43.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:44.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:44.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:44.897 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:52:44.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:52:44.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:44.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:52:45.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:45.129 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:45.119+0000 7f02b5c5d8c0 -1 Falling back to public interface 2026-02-17T11:52:46.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:46.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:46.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:52:46.053 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:52:46.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:46.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:52:46.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:47.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:47.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:47.206 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:52:47.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:52:47.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:47.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:52:47.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:48.099 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:52:48.087+0000 7f02b5c5d8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T11:52:48.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:48.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:48.360 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:52:48.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:52:48.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:48.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:52:48.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:52:49.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:52:49.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:52:49.514 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:52:49.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:52:49.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:52:49.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:52:49.667 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3093199501,v1:127.0.0.1:6819/3093199501] [v2:127.0.0.1:6820/3093199501,v1:127.0.0.1:6821/3093199501] exists,up f7cd5d50-45a2-4c55-a175-2991537e538f 2026-02-17T11:52:49.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:52:49.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:52:49.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:52:49.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3606: corrupt_scrub_erasure: create_rbd_pool 2026-02-17T11:52:49.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T11:52:49.814 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' does not exist 2026-02-17T11:52:49.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T11:52:49.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T11:52:50.024 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T11:52:50.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:52:51.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T11:52:51.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3607: corrupt_scrub_erasure: create_pool foo 1 2026-02-17T11:52:51.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 1 2026-02-17T11:52:51.516 INFO:tasks.workunit.client.0.trial128.stderr:pool 'foo' created 2026-02-17T11:52:51.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:52:52.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3609: corrupt_scrub_erasure: create_ec_pool ecpool false k=2 m=1 stripe_unit=2K --force 2026-02-17T11:52:52.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T11:52:52.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T11:52:52.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=false 2026-02-17T11:52:52.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T11:52:52.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 stripe_unit=2K --force 2026-02-17T11:52:52.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T11:52:52.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T11:52:53.034 INFO:tasks.workunit.client.0.trial128.stderr:pool 'ecpool' created 2026-02-17T11:52:53.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:52:54.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' false = true ']' 2026-02-17T11:52:54.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T11:52:54.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:52:54.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:52:54.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:52:54.049 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:52:54.049 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:52:54.050 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:52:54.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:52:54.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:52:54.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:52:54.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:52:54.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:52:54.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:52:54.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:52:54.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:52:54.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:52:54.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:52:54.311 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:52:54.311 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:52:54.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:52:54.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:52:54.311 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:52:54.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836486 2026-02-17T11:52:54.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836486 2026-02-17T11:52:54.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486' 2026-02-17T11:52:54.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:52:54.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:52:54.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705668 2026-02-17T11:52:54.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705668 2026-02-17T11:52:54.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-38654705668' 2026-02-17T11:52:54.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:52:54.456 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:52:54.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542147 2026-02-17T11:52:54.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542147 2026-02-17T11:52:54.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-38654705668 2-60129542147' 2026-02-17T11:52:54.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:52:54.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836486 2026-02-17T11:52:54.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:52:54.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:52:54.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836486 2026-02-17T11:52:54.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:52:54.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836486 2026-02-17T11:52:54.530 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836486 2026-02-17T11:52:54.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-17T11:52:54.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:52:54.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-17T11:52:54.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:52:55.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:52:55.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:52:55.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-17T11:52:55.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:52:55.851 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705668 2026-02-17T11:52:55.851 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:52:55.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:52:55.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705668 2026-02-17T11:52:55.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:52:55.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705668 2026-02-17T11:52:55.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-17T11:52:55.855 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705668 2026-02-17T11:52:55.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:52:56.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705669 -lt 38654705668 2026-02-17T11:52:56.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:52:56.021 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542147 2026-02-17T11:52:56.021 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:52:56.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:52:56.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542147 2026-02-17T11:52:56.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:52:56.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542147 2026-02-17T11:52:56.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-17T11:52:56.025 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542147 2026-02-17T11:52:56.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:52:56.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-17T11:52:56.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:52:56.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:52:56.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:52:56.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:52:56.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:52:56.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:52:56.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:52:56.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:52:56.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:52:56.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:52:56.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:52:56.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:52:56.507 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:52:56.507 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:52:56.507 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:52:56.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:52:56.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:52:56.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:52:56.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T11:52:56.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3610: corrupt_scrub_erasure: wait_for_clean 2026-02-17T11:52:56.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:52:56.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:52:56.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:52:56.686 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:52:56.686 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:52:56.687 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:52:56.687 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:52:56.687 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:52:56.687 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:52:56.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:52:56.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:52:56.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:52:56.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:52:56.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:52:56.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:52:56.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:52:56.948 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:52:56.948 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:52:56.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:52:56.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:52:56.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:52:57.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836488 2026-02-17T11:52:57.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836488 2026-02-17T11:52:57.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488' 2026-02-17T11:52:57.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:52:57.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:52:57.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705670 2026-02-17T11:52:57.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705670 2026-02-17T11:52:57.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-38654705670' 2026-02-17T11:52:57.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:52:57.091 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:52:57.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542148 2026-02-17T11:52:57.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542148 2026-02-17T11:52:57.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-38654705670 2-60129542148' 2026-02-17T11:52:57.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:52:57.165 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836488 2026-02-17T11:52:57.165 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:52:57.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:52:57.167 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836488 2026-02-17T11:52:57.167 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:52:57.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836488 2026-02-17T11:52:57.169 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836488 2026-02-17T11:52:57.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-17T11:52:57.169 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:52:57.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836488 2026-02-17T11:52:57.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:52:58.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:52:58.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:52:58.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-17T11:52:58.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:52:58.474 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705670 2026-02-17T11:52:58.474 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:52:58.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:52:58.476 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705670 2026-02-17T11:52:58.476 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:52:58.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705670 2026-02-17T11:52:58.477 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705670 2026-02-17T11:52:58.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-17T11:52:58.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:52:58.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705670 -lt 38654705670 2026-02-17T11:52:58.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:52:58.629 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542148 2026-02-17T11:52:58.629 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:52:58.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:52:58.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542148 2026-02-17T11:52:58.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:52:58.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542148 2026-02-17T11:52:58.632 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542148 2026-02-17T11:52:58.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-17T11:52:58.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:52:58.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-17T11:52:58.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:52:58.786 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:52:58.786 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:52:58.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:52:58.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:52:58.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:52:58.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:52:58.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:52:58.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:52:58.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:52:58.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:52:59.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:52:59.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:52:59.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:52:59.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:52:59.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:52:59.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:52:59.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:52:59.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: seq 1 7 2026-02-17T11:52:59.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T11:52:59.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ1 2026-02-17T11:52:59.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ1 2026-02-17T11:52:59.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:52:59.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:52:59.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ1 2026-02-17T11:52:59.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:52:59.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:52:59.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:52:59.540 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:52:59.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:52:59.741 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:52:59.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:52:59.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:52:59.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:52:59.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 1 % 2 2026-02-17T11:52:59.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=1 2026-02-17T11:52:59.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T11:52:59.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3621: corrupt_scrub_erasure: local payload=UVWXYZZZ 2026-02-17T11:52:59.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3622: corrupt_scrub_erasure: echo UVWXYZZZ 2026-02-17T11:52:59.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3623: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:52:59.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:52:59.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:52:59.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:52:59.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:52:59.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:52:59.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:52:59.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:52:59.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:52:59.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:52:59.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:52:59.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:52:59.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:52:59.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:52:59.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:52:59.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:53:01.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:53:01.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:53:01.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:53:01.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:53:01.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:53:01.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:53:01.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:53:01.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:53:02.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:53:02.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:53:02.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:53:02.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:53:02.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:53:02.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:53:02.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:53:02.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:53:02.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:53:02.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:53:02.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:53:02.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:53:02.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:53:02.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:53:02.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:53:02.274 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:53:02.274 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:53:02.274 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:53:02.274 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:53:02.274 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:53:02.274 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:53:02.274 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:53:02.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:53:02.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:53:02.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:53:02.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:53:02.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:53:02.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:53:02.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:53:02.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:53:02.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:53:02.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:53:02.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:53:02.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:53:02.277 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:53:02.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:53:02.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:53:02.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:53:02.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:53:02.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:53:02.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:53:02.296 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:02.283+0000 7f2df626c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:02.297 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:02.287+0000 7f2df626c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:02.298 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:02.287+0000 7f2df626c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:02.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:53:02.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:53:02.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:53:02.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:53:02.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:53:02.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:02.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:53:02.430 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:53:02.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:02.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:53:02.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:03.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:03.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:03.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:53:03.582 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:53:03.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:03.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:53:03.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:03.785 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:03.775+0000 7f2df626c8c0 -1 Falling back to public interface 2026-02-17T11:53:04.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:04.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:04.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:53:04.731 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:53:04.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:04.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:53:04.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:05.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:05.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:05.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:53:05.890 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:53:05.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:05.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:53:06.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:06.797 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:06.787+0000 7f2df626c8c0 -1 osd.1 32 log_to_monitors true 2026-02-17T11:53:07.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:07.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:07.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:53:07.049 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:53:07.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:07.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:53:07.206 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 36 up_thru 36 down_at 33 last_clean_interval [9,32) [v2:127.0.0.1:6810/3482877896,v1:127.0.0.1:6811/3482877896] [v2:127.0.0.1:6812/3482877896,v1:127.0.0.1:6813/3482877896] exists,up 5280bdd0-cca7-4c9e-8ce1-9ce91b6db0a3 2026-02-17T11:53:07.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:53:07.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:53:07.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:53:07.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:53:07.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:53:07.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:53:07.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:53:07.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:53:07.207 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:53:07.208 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:53:07.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:53:07.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:53:07.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:53:07.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:53:07.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:53:07.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:53:07.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:53:07.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:53:07.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:53:07.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:53:07.476 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:53:07.476 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:53:07.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:53:07.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:53:07.476 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:53:07.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836491 2026-02-17T11:53:07.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836491 2026-02-17T11:53:07.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491' 2026-02-17T11:53:07.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:53:07.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:53:07.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822658 2026-02-17T11:53:07.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822658 2026-02-17T11:53:07.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-154618822658' 2026-02-17T11:53:07.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:53:07.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:53:07.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542151 2026-02-17T11:53:07.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542151 2026-02-17T11:53:07.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-154618822658 2-60129542151' 2026-02-17T11:53:07.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:53:07.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836491 2026-02-17T11:53:07.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:53:07.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:53:07.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836491 2026-02-17T11:53:07.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:53:07.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836491 2026-02-17T11:53:07.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-17T11:53:07.698 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836491 2026-02-17T11:53:07.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:53:07.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-17T11:53:07.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:53:07.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-154618822658 2026-02-17T11:53:07.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:53:07.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:53:07.865 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-154618822658 2026-02-17T11:53:07.865 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:53:07.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822658 2026-02-17T11:53:07.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 154618822658' 2026-02-17T11:53:07.867 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 154618822658 2026-02-17T11:53:07.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:53:08.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822658 -lt 154618822658 2026-02-17T11:53:08.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:53:08.027 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542151 2026-02-17T11:53:08.027 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:53:08.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:53:08.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542151 2026-02-17T11:53:08.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:53:08.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542151 2026-02-17T11:53:08.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542151' 2026-02-17T11:53:08.031 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542151 2026-02-17T11:53:08.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:53:08.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542150 -lt 60129542151 2026-02-17T11:53:08.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:53:09.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:53:09.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:53:09.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542150 -lt 60129542151 2026-02-17T11:53:09.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:53:10.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:53:10.350 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:53:10.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542152 -lt 60129542151 2026-02-17T11:53:10.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:53:10.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:53:10.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:53:10.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:53:10.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:53:10.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:53:10.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:53:10.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:53:10.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:53:10.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:53:10.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:53:10.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:53:10.851 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:53:10.851 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:53:10.851 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:53:11.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:53:11.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:53:11.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:53:11.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T11:53:11.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ2 2026-02-17T11:53:11.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ2 2026-02-17T11:53:11.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:53:11.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:53:11.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ2 2026-02-17T11:53:11.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:53:11.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:53:11.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:53:11.231 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:53:11.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:53:11.435 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:53:11.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:53:11.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:53:11.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ2 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:53:11.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 2 % 2 2026-02-17T11:53:11.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=0 2026-02-17T11:53:11.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T11:53:11.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3628: corrupt_scrub_erasure: dd if=/dev/urandom of=td/osd-scrub-repair/CORRUPT bs=2048 count=1 2026-02-17T11:53:11.472 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records in 2026-02-17T11:53:11.472 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records out 2026-02-17T11:53:11.472 INFO:tasks.workunit.client.0.trial128.stderr:2048 bytes (2.0 kB, 2.0 KiB) copied, 8.7452e-05 s, 23.4 MB/s 2026-02-17T11:53:11.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3629: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:53:11.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:53:11.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:53:11.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:53:11.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:53:11.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:53:11.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:53:11.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:53:11.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:53:11.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:53:11.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:53:11.474 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:53:11.474 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:53:11.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:53:11.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:53:11.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:53:12.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:53:12.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:53:12.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:53:12.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:53:12.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:53:12.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:53:12.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:53:12.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:53:13.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:53:13.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:53:13.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:53:13.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:53:13.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:53:13.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:53:13.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:53:13.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:53:13.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:53:13.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:53:13.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:53:13.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:53:13.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:53:13.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:53:13.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:53:13.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:53:13.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:53:13.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:53:13.963 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:53:13.963 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:53:13.963 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:53:13.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:53:13.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:53:13.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:53:13.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:53:13.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:53:13.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:53:13.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:53:13.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:53:13.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:53:13.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:53:13.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:53:13.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:53:13.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:53:13.965 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:53:13.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:53:13.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:53:13.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:53:13.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:53:13.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:53:13.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:53:13.989 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:13.979+0000 7fecd6da18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:13.990 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:13.979+0000 7fecd6da18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:13.991 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:13.979+0000 7fecd6da18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:14.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:53:14.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:53:14.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:53:14.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:53:14.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:53:14.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:14.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:53:14.121 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:53:14.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:14.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:53:14.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:15.214 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:15.203+0000 7fecd6da18c0 -1 Falling back to public interface 2026-02-17T11:53:15.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:15.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:15.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:53:15.273 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:53:15.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:15.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:53:15.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:16.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:16.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:16.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:53:16.423 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:53:16.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:16.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:53:16.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:17.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:17.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:17.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:53:17.574 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:53:17.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:17.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:53:17.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:18.204 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:18.191+0000 7fecd6da18c0 -1 osd.0 39 log_to_monitors true 2026-02-17T11:53:18.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:18.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:18.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:53:18.723 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:53:18.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:18.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:53:18.871 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 43 up_thru 43 down_at 40 last_clean_interval [5,39) [v2:127.0.0.1:6802/3123759594,v1:127.0.0.1:6803/3123759594] [v2:127.0.0.1:6804/3123759594,v1:127.0.0.1:6805/3123759594] exists,up b7951352-7c09-4b75-abda-71432d2b706a 2026-02-17T11:53:18.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:53:18.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:53:18.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:53:18.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:53:18.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:53:18.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:53:18.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:53:18.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:53:18.873 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:53:18.874 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:53:18.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:53:18.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:53:18.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:53:18.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:53:18.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:53:18.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:53:18.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:53:18.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:53:18.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:53:19.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:53:19.140 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:53:19.140 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:53:19.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:53:19.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:53:19.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:53:19.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=184683593730 2026-02-17T11:53:19.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 184683593730 2026-02-17T11:53:19.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-184683593730' 2026-02-17T11:53:19.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:53:19.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:53:19.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822661 2026-02-17T11:53:19.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822661 2026-02-17T11:53:19.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-184683593730 1-154618822661' 2026-02-17T11:53:19.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:53:19.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:53:19.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542155 2026-02-17T11:53:19.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542155 2026-02-17T11:53:19.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-184683593730 1-154618822661 2-60129542155' 2026-02-17T11:53:19.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:53:19.355 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-184683593730 2026-02-17T11:53:19.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:53:19.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:53:19.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-184683593730 2026-02-17T11:53:19.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:53:19.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=184683593730 2026-02-17T11:53:19.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 184683593730' 2026-02-17T11:53:19.359 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 184683593730 2026-02-17T11:53:19.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:53:19.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 184683593730 2026-02-17T11:53:19.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:53:20.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:53:20.519 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:53:20.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 184683593730 -lt 184683593730 2026-02-17T11:53:20.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:53:20.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-154618822661 2026-02-17T11:53:20.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:53:20.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:53:20.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-154618822661 2026-02-17T11:53:20.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:53:20.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822661 2026-02-17T11:53:20.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 154618822661' 2026-02-17T11:53:20.681 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 154618822661 2026-02-17T11:53:20.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:53:20.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822661 -lt 154618822661 2026-02-17T11:53:20.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:53:20.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542155 2026-02-17T11:53:20.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:53:20.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:53:20.837 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542155 2026-02-17T11:53:20.837 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:53:20.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542155 2026-02-17T11:53:20.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542155' 2026-02-17T11:53:20.838 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542155 2026-02-17T11:53:20.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:53:21.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542155 -lt 60129542155 2026-02-17T11:53:21.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:53:21.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:53:21.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:53:21.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:53:21.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:53:21.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:53:21.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:53:21.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:53:21.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:53:21.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:53:21.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:53:21.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:53:21.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:53:21.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:53:21.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:53:21.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:53:21.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:53:21.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:53:21.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T11:53:21.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ3 2026-02-17T11:53:21.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ3 2026-02-17T11:53:21.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:53:21.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:53:21.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ3 2026-02-17T11:53:21.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:53:21.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:53:21.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:53:21.727 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:53:21.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:53:21.931 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:53:21.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:53:21.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:53:21.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ3 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:53:21.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 3 % 2 2026-02-17T11:53:21.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=1 2026-02-17T11:53:21.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T11:53:21.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3634: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ3 remove 2026-02-17T11:53:21.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:53:21.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:53:21.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:53:21.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:53:21.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ3 remove 2026-02-17T11:53:21.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:53:21.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:53:21.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:53:21.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:53:21.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:53:21.974 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:53:21.974 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:53:21.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:53:21.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:53:21.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:53:23.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:53:23.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ3 remove 2026-02-17T11:53:23.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:53:23.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:53:23.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:53:23.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:53:23.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:53:23.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ3 remove 2026-02-17T11:53:23.909 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#3:b197b25d:::EOBJ3:head# 2026-02-17T11:53:24.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:53:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:53:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:53:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:53:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:53:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:53:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:53:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:53:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:53:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:53:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:53:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:53:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:53:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:53:24.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:53:24.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:53:24.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:53:24.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:53:24.447 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:53:24.448 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:53:24.448 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:53:24.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:53:24.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:53:24.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:53:24.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:53:24.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:53:24.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:53:24.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:53:24.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:53:24.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:53:24.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:53:24.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:53:24.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:53:24.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:53:24.451 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:53:24.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:53:24.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:53:24.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:53:24.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:53:24.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:53:24.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:53:24.472 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:24.458+0000 7fc66740c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:24.473 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:24.462+0000 7fc66740c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:24.474 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:24.462+0000 7fc66740c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:24.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:53:24.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:53:24.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:53:24.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:53:24.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:53:24.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:24.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:53:24.606 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:53:24.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:24.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:53:24.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:25.722 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:25.710+0000 7fc66740c8c0 -1 Falling back to public interface 2026-02-17T11:53:25.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:25.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:25.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:53:25.757 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:53:25.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:25.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:53:25.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:26.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:26.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:26.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:53:26.907 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:53:26.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:26.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:53:27.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:28.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:28.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:28.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:53:28.059 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:53:28.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:28.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:53:28.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:28.735 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:28.722+0000 7fc66740c8c0 -1 osd.1 47 log_to_monitors true 2026-02-17T11:53:29.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:29.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:29.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:53:29.210 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:53:29.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:29.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:53:29.358 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 51 up_thru 51 down_at 48 last_clean_interval [36,47) [v2:127.0.0.1:6810/93122792,v1:127.0.0.1:6811/93122792] [v2:127.0.0.1:6812/93122792,v1:127.0.0.1:6813/93122792] exists,up 5280bdd0-cca7-4c9e-8ce1-9ce91b6db0a3 2026-02-17T11:53:29.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:53:29.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:53:29.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:53:29.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:53:29.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:53:29.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:53:29.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:53:29.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:53:29.360 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:53:29.360 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:53:29.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:53:29.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:53:29.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:53:29.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:53:29.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:53:29.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:53:29.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:53:29.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:53:29.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:53:29.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:53:29.622 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:53:29.622 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:53:29.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:53:29.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:53:29.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:53:29.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=184683593733 2026-02-17T11:53:29.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 184683593733 2026-02-17T11:53:29.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-184683593733' 2026-02-17T11:53:29.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:53:29.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:53:29.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=219043332098 2026-02-17T11:53:29.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 219043332098 2026-02-17T11:53:29.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-184683593733 1-219043332098' 2026-02-17T11:53:29.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:53:29.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:53:29.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542158 2026-02-17T11:53:29.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542158 2026-02-17T11:53:29.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-184683593733 1-219043332098 2-60129542158' 2026-02-17T11:53:29.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:53:29.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-184683593733 2026-02-17T11:53:29.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:53:29.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:53:29.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-184683593733 2026-02-17T11:53:29.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:53:29.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=184683593733 2026-02-17T11:53:29.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 184683593733' 2026-02-17T11:53:29.848 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 184683593733 2026-02-17T11:53:29.848 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:53:30.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 184683593732 -lt 184683593733 2026-02-17T11:53:30.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:53:31.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:53:31.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:53:31.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 184683593732 -lt 184683593733 2026-02-17T11:53:31.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:53:32.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:53:32.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:53:32.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 184683593733 -lt 184683593733 2026-02-17T11:53:32.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:53:32.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-219043332098 2026-02-17T11:53:32.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:53:32.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:53:32.312 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-219043332098 2026-02-17T11:53:32.312 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:53:32.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=219043332098 2026-02-17T11:53:32.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 219043332098' 2026-02-17T11:53:32.314 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 219043332098 2026-02-17T11:53:32.314 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:53:32.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 219043332098 -lt 219043332098 2026-02-17T11:53:32.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:53:32.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542158 2026-02-17T11:53:32.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:53:32.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:53:32.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542158 2026-02-17T11:53:32.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:53:32.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542158 2026-02-17T11:53:32.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542158' 2026-02-17T11:53:32.471 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542158 2026-02-17T11:53:32.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:53:32.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542158 -lt 60129542158 2026-02-17T11:53:32.636 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:53:32.636 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:53:32.637 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:53:32.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:53:32.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:53:32.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:53:32.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:53:32.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:53:32.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:53:32.818 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:53:32.818 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:53:32.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:53:32.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:53:32.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:53:32.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:53:33.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:53:33.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:53:33.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:53:33.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T11:53:33.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ4 2026-02-17T11:53:33.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ4 2026-02-17T11:53:33.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:53:33.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:53:33.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ4 2026-02-17T11:53:33.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:53:33.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:53:33.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:53:33.344 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:53:33.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:53:33.549 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:53:33.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:53:33.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:53:33.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ4 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:53:33.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 4 % 2 2026-02-17T11:53:33.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=0 2026-02-17T11:53:33.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T11:53:33.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3638: corrupt_scrub_erasure: rados --pool ecpool setxattr EOBJ4 key1-EOBJ4 val1-EOBJ4 2026-02-17T11:53:33.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3639: corrupt_scrub_erasure: rados --pool ecpool setxattr EOBJ4 key2-EOBJ4 val2-EOBJ4 2026-02-17T11:53:33.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3642: corrupt_scrub_erasure: echo -n bad-val 2026-02-17T11:53:33.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3643: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-17T11:53:33.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:53:33.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:53:33.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:53:33.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:53:33.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-17T11:53:33.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:53:33.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:53:33.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:53:33.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:53:33.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:53:33.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:53:33.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:53:33.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:53:33.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:53:33.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:53:34.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:53:34.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-17T11:53:34.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:53:34.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:53:34.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:53:34.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:53:34.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:53:34.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-17T11:53:36.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:53:36.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:53:36.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:53:36.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:53:36.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:53:36.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:53:36.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:53:36.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:53:36.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:53:36.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:53:36.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:53:36.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:53:36.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:53:36.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:53:36.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:53:36.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:53:36.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:53:36.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:53:36.126 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:53:36.126 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:53:36.127 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:53:36.127 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:53:36.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:53:36.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:53:36.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:53:36.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:53:36.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:53:36.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:53:36.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:53:36.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:53:36.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:53:36.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:53:36.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:53:36.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:53:36.129 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:53:36.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:53:36.129 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:53:36.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:53:36.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:53:36.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:53:36.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:53:36.148 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:36.138+0000 7fcd7a8dc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:36.150 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:36.138+0000 7fcd7a8dc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:36.151 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:36.138+0000 7fcd7a8dc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:36.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:53:36.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:53:36.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:53:36.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:53:36.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:53:36.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:36.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:53:36.279 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:53:36.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:36.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:53:36.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:37.398 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:37.386+0000 7fcd7a8dc8c0 -1 Falling back to public interface 2026-02-17T11:53:37.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:37.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:37.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:53:37.436 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:53:37.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:37.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:53:37.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:38.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:38.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:38.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:53:38.590 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:53:38.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:38.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:53:38.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:39.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:39.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:39.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:53:39.750 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:53:39.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:39.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:53:39.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:40.406 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:40.394+0000 7fcd7a8dc8c0 -1 osd.0 54 log_to_monitors true 2026-02-17T11:53:40.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:40.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:40.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:53:40.906 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:53:40.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:40.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:53:41.063 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 58 up_thru 58 down_at 55 last_clean_interval [43,54) [v2:127.0.0.1:6802/1335342710,v1:127.0.0.1:6803/1335342710] [v2:127.0.0.1:6804/1335342710,v1:127.0.0.1:6805/1335342710] exists,up b7951352-7c09-4b75-abda-71432d2b706a 2026-02-17T11:53:41.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:53:41.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:53:41.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:53:41.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:53:41.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:53:41.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:53:41.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:53:41.065 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:53:41.065 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:53:41.065 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:53:41.065 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:53:41.065 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:53:41.065 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:53:41.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:53:41.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:53:41.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:53:41.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:53:41.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:53:41.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:53:41.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:53:41.332 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:53:41.332 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:53:41.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:53:41.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:53:41.333 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:53:41.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=249108103170 2026-02-17T11:53:41.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 249108103170 2026-02-17T11:53:41.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103170' 2026-02-17T11:53:41.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:53:41.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:53:41.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=219043332101 2026-02-17T11:53:41.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 219043332101 2026-02-17T11:53:41.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103170 1-219043332101' 2026-02-17T11:53:41.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:53:41.476 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:53:41.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542161 2026-02-17T11:53:41.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542161 2026-02-17T11:53:41.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103170 1-219043332101 2-60129542161' 2026-02-17T11:53:41.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:53:41.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-249108103170 2026-02-17T11:53:41.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:53:41.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:53:41.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-249108103170 2026-02-17T11:53:41.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:53:41.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=249108103170 2026-02-17T11:53:41.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 249108103170' 2026-02-17T11:53:41.550 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 249108103170 2026-02-17T11:53:41.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:53:41.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 184683593734 -lt 249108103170 2026-02-17T11:53:41.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:53:42.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:53:42.715 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:53:42.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 249108103170 -lt 249108103170 2026-02-17T11:53:42.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:53:42.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-219043332101 2026-02-17T11:53:42.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:53:42.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:53:42.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-219043332101 2026-02-17T11:53:42.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:53:42.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=219043332101 2026-02-17T11:53:42.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 219043332101' 2026-02-17T11:53:42.873 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 219043332101 2026-02-17T11:53:42.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:53:43.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 219043332101 -lt 219043332101 2026-02-17T11:53:43.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:53:43.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542161 2026-02-17T11:53:43.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:53:43.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:53:43.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542161 2026-02-17T11:53:43.030 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:53:43.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542161 2026-02-17T11:53:43.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542161' 2026-02-17T11:53:43.031 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542161 2026-02-17T11:53:43.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:53:43.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542161 -lt 60129542161 2026-02-17T11:53:43.189 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:53:43.189 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:53:43.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:53:43.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:53:43.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:53:43.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:53:43.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:53:43.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:53:43.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:53:43.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:53:43.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:53:43.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:53:43.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:53:43.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:53:43.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:53:43.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:53:43.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:53:43.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:53:43.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3644: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-17T11:53:43.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:53:43.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:53:43.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T11:53:43.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:53:43.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-17T11:53:43.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:53:43.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:53:43.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T11:53:43.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:53:43.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T11:53:43.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:53:43.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:53:43.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:53:43.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:53:43.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:53:45.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:53:45.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-17T11:53:45.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:53:45.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:53:45.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T11:53:45.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:53:45.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:53:45.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-17T11:53:46.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T11:53:46.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:53:46.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:53:46.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T11:53:46.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:53:46.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:53:46.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:53:46.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:53:46.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:53:46.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:53:46.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:53:46.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:53:46.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:53:46.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:53:46.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:53:46.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:53:46.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:53:46.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:53:46.185 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:53:46.185 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:53:46.185 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:53:46.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:53:46.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:53:46.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:53:46.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:53:46.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:53:46.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:53:46.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:53:46.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:53:46.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:53:46.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:53:46.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:53:46.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:53:46.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T11:53:46.188 INFO:tasks.workunit.client.0.trial128.stderr:start osd.2 2026-02-17T11:53:46.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:53:46.188 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T11:53:46.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T11:53:46.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:53:46.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:53:46.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:53:46.208 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:46.198+0000 7fb0a9de48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:46.210 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:46.198+0000 7fb0a9de48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:46.211 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:46.198+0000 7fb0a9de48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:46.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T11:53:46.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:53:46.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:53:46.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:53:46.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:53:46.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:46.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:53:46.345 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:53:46.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:46.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:53:46.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:47.458 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:47.446+0000 7fb0a9de48c0 -1 Falling back to public interface 2026-02-17T11:53:47.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:47.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:47.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:53:47.496 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:53:47.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:47.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:53:47.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:48.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:48.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:48.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:53:48.643 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:53:48.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:48.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:53:48.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:49.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:49.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:49.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:53:49.795 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:53:49.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:49.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:53:49.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:50.908 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:50.894+0000 7fb0a9de48c0 -1 osd.2 59 log_to_monitors true 2026-02-17T11:53:50.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:50.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:50.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:53:50.947 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:53:50.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:50.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:53:51.095 INFO:tasks.workunit.client.0.trial128.stderr:osd.2 up in weight 1 up_from 63 up_thru 58 down_at 60 last_clean_interval [14,59) [v2:127.0.0.1:6818/666714889,v1:127.0.0.1:6819/666714889] [v2:127.0.0.1:6820/666714889,v1:127.0.0.1:6821/666714889] exists,up f7cd5d50-45a2-4c55-a175-2991537e538f 2026-02-17T11:53:51.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:53:51.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:53:51.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:53:51.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:53:51.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:53:51.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:53:51.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:53:51.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:53:51.096 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:53:51.096 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:53:51.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:53:51.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:53:51.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:53:51.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:53:51.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:53:51.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:53:51.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:53:51.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:53:51.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:53:51.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:53:51.360 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:53:51.360 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:53:51.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:53:51.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:53:51.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:53:51.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=249108103173 2026-02-17T11:53:51.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 249108103173 2026-02-17T11:53:51.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103173' 2026-02-17T11:53:51.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:53:51.430 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:53:51.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=219043332104 2026-02-17T11:53:51.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 219043332104 2026-02-17T11:53:51.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103173 1-219043332104' 2026-02-17T11:53:51.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:53:51.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:53:51.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=270582939650 2026-02-17T11:53:51.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 270582939650 2026-02-17T11:53:51.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103173 1-219043332104 2-270582939650' 2026-02-17T11:53:51.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:53:51.574 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-249108103173 2026-02-17T11:53:51.574 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:53:51.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:53:51.576 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-249108103173 2026-02-17T11:53:51.576 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:53:51.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=249108103173 2026-02-17T11:53:51.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 249108103173' 2026-02-17T11:53:51.577 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 249108103173 2026-02-17T11:53:51.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:53:51.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 249108103171 -lt 249108103173 2026-02-17T11:53:51.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:53:52.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:53:52.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:53:52.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 249108103173 -lt 249108103173 2026-02-17T11:53:52.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:53:52.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-219043332104 2026-02-17T11:53:52.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:53:52.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:53:52.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-219043332104 2026-02-17T11:53:52.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:53:52.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=219043332104 2026-02-17T11:53:52.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 219043332104' 2026-02-17T11:53:52.893 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 219043332104 2026-02-17T11:53:52.893 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:53:53.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 219043332104 -lt 219043332104 2026-02-17T11:53:53.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:53:53.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-270582939650 2026-02-17T11:53:53.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:53:53.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:53:53.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-270582939650 2026-02-17T11:53:53.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:53:53.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=270582939650 2026-02-17T11:53:53.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 270582939650' 2026-02-17T11:53:53.055 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 270582939650 2026-02-17T11:53:53.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:53:53.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 270582939650 -lt 270582939650 2026-02-17T11:53:53.218 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:53:53.218 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:53:53.218 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:53:53.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:53:53.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:53:53.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:53:53.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:53:53.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:53:53.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:53:53.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:53:53.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:53:53.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:53:53.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:53:53.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:53:53.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:53:53.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:53:53.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:53:53.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:53:53.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3645: corrupt_scrub_erasure: echo -n val3-EOBJ4 2026-02-17T11:53:53.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3646: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-17T11:53:53.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:53:53.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:53:53.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T11:53:53.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:53:53.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-17T11:53:53.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:53:53.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:53:53.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T11:53:53.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:53:53.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T11:53:53.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:53:53.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:53:53.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:53:53.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:53:53.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:53:55.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:53:55.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-17T11:53:55.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:53:55.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:53:55.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T11:53:55.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:53:55.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:53:55.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-17T11:53:56.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T11:53:56.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:53:56.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:53:56.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T11:53:56.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:53:56.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:53:56.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:53:56.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:53:56.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:53:56.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:53:56.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:53:56.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:53:56.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:53:56.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:53:56.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:53:56.229 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:53:56.229 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:53:56.229 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:53:56.229 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:53:56.229 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:53:56.229 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:53:56.229 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:53:56.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:53:56.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:53:56.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:53:56.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:53:56.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:53:56.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:53:56.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:53:56.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:53:56.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:53:56.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:53:56.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:53:56.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T11:53:56.232 INFO:tasks.workunit.client.0.trial128.stderr:start osd.2 2026-02-17T11:53:56.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:53:56.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T11:53:56.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T11:53:56.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:53:56.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:53:56.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:53:56.252 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:56.238+0000 7fae745048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:56.253 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:56.242+0000 7fae745048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:56.254 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:56.242+0000 7fae745048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:53:56.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T11:53:56.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:53:56.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:53:56.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:53:56.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:53:56.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:56.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:53:56.386 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:53:56.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:56.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:53:56.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:57.506 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:53:57.494+0000 7fae745048c0 -1 Falling back to public interface 2026-02-17T11:53:57.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:57.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:57.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:53:57.540 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:53:57.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:57.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:53:57.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:58.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:58.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:58.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:53:58.691 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:53:58.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:58.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:53:58.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:53:59.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:53:59.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:53:59.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:53:59.842 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:53:59.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:53:59.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:53:59.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:00.516 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:00.506+0000 7fae745048c0 -1 osd.2 64 log_to_monitors true 2026-02-17T11:54:00.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:00.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:00.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:54:00.992 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:54:00.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:00.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:54:01.141 INFO:tasks.workunit.client.0.trial128.stderr:osd.2 up in weight 1 up_from 68 up_thru 68 down_at 65 last_clean_interval [63,64) [v2:127.0.0.1:6818/682816047,v1:127.0.0.1:6819/682816047] [v2:127.0.0.1:6820/682816047,v1:127.0.0.1:6821/682816047] exists,up f7cd5d50-45a2-4c55-a175-2991537e538f 2026-02-17T11:54:01.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:54:01.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:54:01.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:54:01.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:54:01.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:54:01.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:54:01.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:54:01.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:54:01.143 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:54:01.143 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:54:01.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:54:01.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:54:01.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:54:01.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:54:01.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:54:01.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:54:01.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:54:01.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:54:01.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:54:01.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:54:01.416 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:54:01.416 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:54:01.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:54:01.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:01.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:54:01.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=249108103176 2026-02-17T11:54:01.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 249108103176 2026-02-17T11:54:01.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103176' 2026-02-17T11:54:01.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:01.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:54:01.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=219043332107 2026-02-17T11:54:01.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 219043332107 2026-02-17T11:54:01.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103176 1-219043332107' 2026-02-17T11:54:01.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:01.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:54:01.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776130 2026-02-17T11:54:01.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776130 2026-02-17T11:54:01.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103176 1-219043332107 2-292057776130' 2026-02-17T11:54:01.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:01.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-249108103176 2026-02-17T11:54:01.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:01.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:54:01.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-249108103176 2026-02-17T11:54:01.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:01.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=249108103176 2026-02-17T11:54:01.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 249108103176' 2026-02-17T11:54:01.635 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 249108103176 2026-02-17T11:54:01.635 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:54:01.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 249108103176 -lt 249108103176 2026-02-17T11:54:01.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:01.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-219043332107 2026-02-17T11:54:01.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:01.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:54:01.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-219043332107 2026-02-17T11:54:01.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:01.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=219043332107 2026-02-17T11:54:01.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 219043332107' 2026-02-17T11:54:01.798 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 219043332107 2026-02-17T11:54:01.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:54:01.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 219043332107 -lt 219043332107 2026-02-17T11:54:01.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:01.961 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-292057776130 2026-02-17T11:54:01.961 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:01.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:54:01.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-292057776130 2026-02-17T11:54:01.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:01.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776130 2026-02-17T11:54:01.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 292057776130' 2026-02-17T11:54:01.964 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 292057776130 2026-02-17T11:54:01.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:54:02.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776130 -lt 292057776130 2026-02-17T11:54:02.111 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:54:02.111 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:54:02.111 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:54:02.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:54:02.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:54:02.287 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:54:02.287 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:54:02.287 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:54:02.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:54:02.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:54:02.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:54:02.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:54:02.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:54:02.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:54:02.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:54:02.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:54:02.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:54:02.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:54:02.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3647: corrupt_scrub_erasure: rm td/osd-scrub-repair/bad-val td/osd-scrub-repair/newval 2026-02-17T11:54:02.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T11:54:02.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ5 2026-02-17T11:54:02.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ5 2026-02-17T11:54:02.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:54:02.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:54:02.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ5 2026-02-17T11:54:02.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:54:02.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:54:02.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:54:02.827 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:54:02.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:54:03.033 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:54:03.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:54:03.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:54:03.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ5 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:54:03.065 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 5 % 2 2026-02-17T11:54:03.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=1 2026-02-17T11:54:03.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T11:54:03.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3652: corrupt_scrub_erasure: dd if=/dev/urandom of=td/osd-scrub-repair/CORRUPT bs=2048 count=2 2026-02-17T11:54:03.067 INFO:tasks.workunit.client.0.trial128.stderr:2+0 records in 2026-02-17T11:54:03.067 INFO:tasks.workunit.client.0.trial128.stderr:2+0 records out 2026-02-17T11:54:03.067 INFO:tasks.workunit.client.0.trial128.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 9.8684e-05 s, 41.5 MB/s 2026-02-17T11:54:03.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3653: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:54:03.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:54:03.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:54:03.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:54:03.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:54:03.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:54:03.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:54:03.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:54:03.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:54:03.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:54:03.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:54:03.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:54:03.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:54:03.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:54:03.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:54:03.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:54:04.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:54:04.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:54:04.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:54:04.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:54:04.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:54:04.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:54:04.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:54:04.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:54:05.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:54:05.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:54:05.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:54:05.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:54:05.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:54:05.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:54:05.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:54:05.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:54:05.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:54:05.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:54:05.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:54:05.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:54:05.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:54:05.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:54:05.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:54:05.585 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:54:05.585 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:54:05.585 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:54:05.585 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:54:05.585 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:54:05.585 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:54:05.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:54:05.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:54:05.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:54:05.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:54:05.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:54:05.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:54:05.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:54:05.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:54:05.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:54:05.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:54:05.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:54:05.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:54:05.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:54:05.587 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:54:05.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:54:05.588 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:54:05.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:54:05.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:54:05.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:54:05.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:54:05.604 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:05.594+0000 7f7e6a59d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:54:05.605 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:05.594+0000 7f7e6a59d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:54:05.606 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:05.594+0000 7f7e6a59d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:54:05.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:54:05.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:54:05.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:54:05.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:54:05.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:54:05.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:05.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:54:05.746 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:54:05.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:05.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:54:05.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:06.838 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:06.826+0000 7f7e6a59d8c0 -1 Falling back to public interface 2026-02-17T11:54:06.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:06.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:06.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:54:06.896 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:54:06.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:06.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:54:07.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:08.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:08.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:08.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:54:08.046 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:54:08.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:08.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:54:08.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:09.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:09.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:09.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:54:09.198 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:54:09.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:09.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:54:09.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:09.825 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:09.814+0000 7f7e6a59d8c0 -1 osd.1 71 log_to_monitors true 2026-02-17T11:54:10.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:10.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:10.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:54:10.349 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:54:10.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:10.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:54:10.499 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 75 up_thru 75 down_at 72 last_clean_interval [51,71) [v2:127.0.0.1:6810/664618687,v1:127.0.0.1:6811/664618687] [v2:127.0.0.1:6812/664618687,v1:127.0.0.1:6813/664618687] exists,up 5280bdd0-cca7-4c9e-8ce1-9ce91b6db0a3 2026-02-17T11:54:10.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:54:10.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:54:10.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:54:10.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:54:10.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:54:10.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:54:10.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:54:10.500 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:54:10.501 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:54:10.501 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:54:10.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:54:10.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:54:10.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:54:10.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:54:10.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:54:10.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:54:10.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:54:10.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:54:10.611 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:54:10.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:54:10.769 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:54:10.769 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:54:10.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:54:10.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:10.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:54:10.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=249108103179 2026-02-17T11:54:10.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 249108103179 2026-02-17T11:54:10.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103179' 2026-02-17T11:54:10.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:10.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:54:10.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=322122547202 2026-02-17T11:54:10.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 322122547202 2026-02-17T11:54:10.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103179 1-322122547202' 2026-02-17T11:54:10.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:10.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:54:10.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776133 2026-02-17T11:54:10.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776133 2026-02-17T11:54:10.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-249108103179 1-322122547202 2-292057776133' 2026-02-17T11:54:10.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:10.986 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-249108103179 2026-02-17T11:54:10.986 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:10.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:54:10.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-249108103179 2026-02-17T11:54:10.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:10.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=249108103179 2026-02-17T11:54:10.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 249108103179' 2026-02-17T11:54:10.990 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 249108103179 2026-02-17T11:54:10.990 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:54:11.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 249108103177 -lt 249108103179 2026-02-17T11:54:11.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:54:12.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:54:12.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:54:12.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 249108103179 -lt 249108103179 2026-02-17T11:54:12.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:12.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-322122547202 2026-02-17T11:54:12.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:12.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:54:12.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-322122547202 2026-02-17T11:54:12.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:12.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=322122547202 2026-02-17T11:54:12.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 322122547202' 2026-02-17T11:54:12.312 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 322122547202 2026-02-17T11:54:12.312 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:54:12.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 322122547202 -lt 322122547202 2026-02-17T11:54:12.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:12.480 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-292057776133 2026-02-17T11:54:12.480 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:12.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:54:12.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-292057776133 2026-02-17T11:54:12.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:12.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776133 2026-02-17T11:54:12.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 292057776133' 2026-02-17T11:54:12.483 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 292057776133 2026-02-17T11:54:12.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:54:12.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776133 -lt 292057776133 2026-02-17T11:54:12.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:54:12.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:54:12.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:54:12.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:54:12.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:54:12.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:54:12.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:54:12.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:54:12.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:54:12.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:54:12.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:54:12.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:54:12.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:54:12.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:54:12.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:54:13.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:54:13.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:54:13.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:54:13.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T11:54:13.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ6 2026-02-17T11:54:13.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ6 2026-02-17T11:54:13.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:54:13.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:54:13.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ6 2026-02-17T11:54:13.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:54:13.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:54:13.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:54:13.346 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:54:13.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:54:13.551 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:54:13.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:54:13.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:54:13.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ6 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:54:13.587 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 6 % 2 2026-02-17T11:54:13.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=0 2026-02-17T11:54:13.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T11:54:13.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3657: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-17T11:54:13.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:54:13.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:54:13.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:54:13.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:54:13.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-17T11:54:13.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:54:13.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:54:13.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:54:13.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:54:13.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:54:13.591 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:54:13.591 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:54:13.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:54:13.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:54:13.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:54:14.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:54:14.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-17T11:54:14.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:54:14.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:54:14.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:54:14.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:54:14.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:54:14.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ6 rm-attr hinfo_key 2026-02-17T11:54:16.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:54:16.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:54:16.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:54:16.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:54:16.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:54:16.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:54:16.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:54:16.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:54:16.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:54:16.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:54:16.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:54:16.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:54:16.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:54:16.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:54:16.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:54:16.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:54:16.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:54:16.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:54:16.097 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:54:16.097 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:54:16.097 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:54:16.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:54:16.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:54:16.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:54:16.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:54:16.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:54:16.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:54:16.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:54:16.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:54:16.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:54:16.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:54:16.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:54:16.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:54:16.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:54:16.098 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:54:16.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:54:16.099 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:54:16.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:54:16.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:54:16.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:54:16.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:54:16.116 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:16.106+0000 7f296831c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:54:16.118 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:16.106+0000 7f296831c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:54:16.119 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:16.106+0000 7f296831c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:54:16.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:54:16.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:54:16.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:54:16.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:54:16.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:54:16.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:16.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:54:16.245 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:54:16.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:16.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:54:16.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:17.386 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:17.374+0000 7f296831c8c0 -1 Falling back to public interface 2026-02-17T11:54:17.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:17.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:17.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:54:17.402 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:54:17.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:17.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:54:17.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:18.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:18.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:18.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:54:18.556 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:54:18.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:18.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:54:18.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:19.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:19.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:19.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:54:19.712 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:54:19.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:19.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:54:19.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:20.402 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:20.390+0000 7f296831c8c0 -1 osd.0 78 log_to_monitors true 2026-02-17T11:54:20.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:20.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:20.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:54:20.872 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:54:20.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:20.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:54:21.025 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 82 up_thru 82 down_at 79 last_clean_interval [58,78) [v2:127.0.0.1:6802/3498282712,v1:127.0.0.1:6803/3498282712] [v2:127.0.0.1:6804/3498282712,v1:127.0.0.1:6805/3498282712] exists,up b7951352-7c09-4b75-abda-71432d2b706a 2026-02-17T11:54:21.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:54:21.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:54:21.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:54:21.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:54:21.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:54:21.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:54:21.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:54:21.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:54:21.027 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:54:21.027 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:54:21.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:54:21.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:54:21.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:54:21.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:54:21.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:54:21.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:54:21.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:54:21.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:54:21.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:54:21.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:54:21.283 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:54:21.283 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:54:21.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:54:21.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:21.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:54:21.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=352187318274 2026-02-17T11:54:21.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 352187318274 2026-02-17T11:54:21.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318274' 2026-02-17T11:54:21.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:21.355 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:54:21.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=322122547205 2026-02-17T11:54:21.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 322122547205 2026-02-17T11:54:21.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318274 1-322122547205' 2026-02-17T11:54:21.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:21.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:54:21.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776136 2026-02-17T11:54:21.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776136 2026-02-17T11:54:21.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318274 1-322122547205 2-292057776136' 2026-02-17T11:54:21.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:21.499 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-352187318274 2026-02-17T11:54:21.499 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:21.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:54:21.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-352187318274 2026-02-17T11:54:21.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:21.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=352187318274 2026-02-17T11:54:21.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 352187318274' 2026-02-17T11:54:21.503 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 352187318274 2026-02-17T11:54:21.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:54:21.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 249108103179 -lt 352187318274 2026-02-17T11:54:21.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:54:22.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:54:22.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:54:22.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318274 -lt 352187318274 2026-02-17T11:54:22.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:22.820 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-322122547205 2026-02-17T11:54:22.820 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:22.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:54:22.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-322122547205 2026-02-17T11:54:22.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:22.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=322122547205 2026-02-17T11:54:22.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 322122547205' 2026-02-17T11:54:22.824 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 322122547205 2026-02-17T11:54:22.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:54:22.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 322122547205 -lt 322122547205 2026-02-17T11:54:22.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:22.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-292057776136 2026-02-17T11:54:22.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:22.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:54:22.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-292057776136 2026-02-17T11:54:22.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:22.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776136 2026-02-17T11:54:22.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 292057776136' 2026-02-17T11:54:22.986 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 292057776136 2026-02-17T11:54:22.986 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:54:23.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776136 -lt 292057776136 2026-02-17T11:54:23.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:54:23.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:54:23.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:54:23.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:54:23.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:54:23.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:54:23.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:54:23.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:54:23.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:54:23.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:54:23.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:54:23.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:54:23.488 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:54:23.489 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:54:23.489 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:54:23.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:54:23.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:54:23.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:54:23.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3658: corrupt_scrub_erasure: echo -n bad-val 2026-02-17T11:54:23.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3659: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-17T11:54:23.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:54:23.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:54:23.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:54:23.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:54:23.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-17T11:54:23.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:54:23.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:54:23.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:54:23.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:54:23.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:54:23.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:54:23.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:54:23.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:54:23.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:54:23.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:54:24.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:54:24.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-17T11:54:24.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:54:24.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:54:24.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:54:24.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:54:24.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:54:24.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-17T11:54:26.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:54:26.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:54:26.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:54:26.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:54:26.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:54:26.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:54:26.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:54:26.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:54:26.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:54:26.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:54:26.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:54:26.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:54:26.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:54:26.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:54:26.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:54:26.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:54:26.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:54:26.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:54:26.186 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:54:26.186 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:54:26.186 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:54:26.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:54:26.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:54:26.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:54:26.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:54:26.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:54:26.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:54:26.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:54:26.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:54:26.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:54:26.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:54:26.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:54:26.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:54:26.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:54:26.188 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:54:26.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:54:26.189 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:54:26.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:54:26.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:54:26.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:54:26.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:54:26.209 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:26.197+0000 7f681d44d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:54:26.210 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:26.197+0000 7f681d44d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:54:26.211 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:26.197+0000 7f681d44d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:54:26.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:54:26.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:54:26.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:54:26.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:54:26.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:54:26.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:26.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:54:26.340 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:54:26.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:26.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:54:26.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:27.474 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:27.465+0000 7f681d44d8c0 -1 Falling back to public interface 2026-02-17T11:54:27.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:27.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:27.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:54:27.492 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:54:27.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:27.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:54:27.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:28.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:28.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:28.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:54:28.643 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:54:28.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:28.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:54:28.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:29.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:29.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:29.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:54:29.794 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:54:29.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:29.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:54:29.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:30.460 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:30.453+0000 7f681d44d8c0 -1 osd.1 83 log_to_monitors true 2026-02-17T11:54:30.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:30.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:30.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:54:30.943 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:54:30.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:30.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:54:31.098 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 87 up_thru 87 down_at 84 last_clean_interval [75,83) [v2:127.0.0.1:6810/3909746176,v1:127.0.0.1:6811/3909746176] [v2:127.0.0.1:6812/3909746176,v1:127.0.0.1:6813/3909746176] exists,up 5280bdd0-cca7-4c9e-8ce1-9ce91b6db0a3 2026-02-17T11:54:31.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:54:31.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:54:31.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:54:31.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:54:31.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:54:31.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:54:31.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:54:31.099 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:54:31.100 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:54:31.100 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:54:31.101 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:54:31.101 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:54:31.101 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:54:31.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:54:31.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:54:31.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:54:31.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:54:31.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:54:31.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:54:31.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:54:31.365 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:54:31.365 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:54:31.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:54:31.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:31.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:54:31.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=352187318277 2026-02-17T11:54:31.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 352187318277 2026-02-17T11:54:31.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318277' 2026-02-17T11:54:31.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:31.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:54:31.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=373662154754 2026-02-17T11:54:31.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 373662154754 2026-02-17T11:54:31.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318277 1-373662154754' 2026-02-17T11:54:31.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:31.507 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:54:31.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776139 2026-02-17T11:54:31.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776139 2026-02-17T11:54:31.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318277 1-373662154754 2-292057776139' 2026-02-17T11:54:31.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:31.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-352187318277 2026-02-17T11:54:31.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:31.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:54:31.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-352187318277 2026-02-17T11:54:31.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:31.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=352187318277 2026-02-17T11:54:31.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 352187318277' 2026-02-17T11:54:31.583 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 352187318277 2026-02-17T11:54:31.583 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:54:31.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318275 -lt 352187318277 2026-02-17T11:54:31.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:54:32.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:54:32.742 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:54:32.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318277 -lt 352187318277 2026-02-17T11:54:32.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:32.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-373662154754 2026-02-17T11:54:32.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:32.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:54:32.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-373662154754 2026-02-17T11:54:32.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:32.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=373662154754 2026-02-17T11:54:32.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 373662154754' 2026-02-17T11:54:32.903 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 373662154754 2026-02-17T11:54:32.903 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:54:33.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 373662154754 -lt 373662154754 2026-02-17T11:54:33.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:33.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-292057776139 2026-02-17T11:54:33.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:33.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:54:33.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-292057776139 2026-02-17T11:54:33.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:33.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776139 2026-02-17T11:54:33.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 292057776139' 2026-02-17T11:54:33.064 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 292057776139 2026-02-17T11:54:33.065 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:54:33.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776139 -lt 292057776139 2026-02-17T11:54:33.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:54:33.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:54:33.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:54:33.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:54:33.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:54:33.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:54:33.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:54:33.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:54:33.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:54:33.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:54:33.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:54:33.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:54:33.557 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:54:33.557 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:54:33.557 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:54:33.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:54:33.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:54:33.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:54:33.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T11:54:33.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ7 2026-02-17T11:54:33.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ7 2026-02-17T11:54:33.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:54:33.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:54:33.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ7 2026-02-17T11:54:33.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:54:33.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:54:33.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:54:33.937 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:54:33.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:54:34.141 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:54:34.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:54:34.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:54:34.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ7 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:54:34.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 7 % 2 2026-02-17T11:54:34.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=1 2026-02-17T11:54:34.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T11:54:34.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3663: corrupt_scrub_erasure: local payload=MAKETHISDIFFERENTFROMOTHEROBJECTS 2026-02-17T11:54:34.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3664: corrupt_scrub_erasure: echo MAKETHISDIFFERENTFROMOTHEROBJECTS 2026-02-17T11:54:34.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3665: corrupt_scrub_erasure: rados --pool ecpool put EOBJ7 td/osd-scrub-repair/DIFFERENT 2026-02-17T11:54:34.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3668: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-17T11:54:34.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:54:34.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:54:34.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:54:34.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:54:34.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-17T11:54:34.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:54:34.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:54:34.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:54:34.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:54:34.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:54:34.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:54:34.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:54:34.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:54:34.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:54:34.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:54:35.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:54:35.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-17T11:54:35.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:54:35.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:54:35.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:54:35.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:54:35.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:54:35.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ1 get-attr hinfo_key 2026-02-17T11:54:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:54:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:54:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:54:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:54:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:54:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:54:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:54:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:54:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:54:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:54:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:54:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:54:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:54:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:54:36.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:54:36.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:54:36.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:54:36.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:54:36.369 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:54:36.369 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:54:36.369 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:54:36.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:54:36.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:54:36.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:54:36.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:54:36.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:54:36.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:54:36.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:54:36.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:54:36.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:54:36.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:54:36.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:54:36.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:54:36.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:54:36.372 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:54:36.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:54:36.372 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:54:36.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:54:36.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:54:36.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:54:36.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:54:36.386 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:36.377+0000 7fb77b3f48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:54:36.388 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:36.381+0000 7fb77b3f48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:54:36.389 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:36.381+0000 7fb77b3f48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:54:36.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:54:36.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:54:36.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:54:36.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:54:36.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:54:36.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:36.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:54:36.524 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:54:36.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:36.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:54:36.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:37.650 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:37.641+0000 7fb77b3f48c0 -1 Falling back to public interface 2026-02-17T11:54:37.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:37.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:37.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:54:37.682 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:54:37.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:37.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:54:37.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:38.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:38.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:38.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:54:38.835 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:54:38.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:38.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:54:38.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:39.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:39.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:39.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:54:39.989 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:54:39.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:39.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:54:40.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:40.653 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:40.645+0000 7fb77b3f48c0 -1 osd.0 90 log_to_monitors true 2026-02-17T11:54:41.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:41.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:41.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:54:41.145 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:54:41.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:41.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:54:41.296 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 94 up_thru 94 down_at 91 last_clean_interval [82,90) [v2:127.0.0.1:6802/1030812105,v1:127.0.0.1:6803/1030812105] [v2:127.0.0.1:6804/1030812105,v1:127.0.0.1:6805/1030812105] exists,up b7951352-7c09-4b75-abda-71432d2b706a 2026-02-17T11:54:41.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:54:41.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:54:41.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:54:41.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:54:41.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:54:41.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:54:41.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:54:41.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:54:41.298 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:54:41.298 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:54:41.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:54:41.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:54:41.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:54:41.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:54:41.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:54:41.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:54:41.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:54:41.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:54:41.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:54:41.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:54:41.562 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:54:41.562 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:54:41.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:54:41.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:41.562 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:54:41.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=403726925826 2026-02-17T11:54:41.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 403726925826 2026-02-17T11:54:41.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-403726925826' 2026-02-17T11:54:41.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:41.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:54:41.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=373662154757 2026-02-17T11:54:41.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 373662154757 2026-02-17T11:54:41.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-403726925826 1-373662154757' 2026-02-17T11:54:41.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:41.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:54:41.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776142 2026-02-17T11:54:41.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776142 2026-02-17T11:54:41.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-403726925826 1-373662154757 2-292057776142' 2026-02-17T11:54:41.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:41.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-403726925826 2026-02-17T11:54:41.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:41.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:54:41.778 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-403726925826 2026-02-17T11:54:41.778 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:41.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=403726925826 2026-02-17T11:54:41.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 403726925826' 2026-02-17T11:54:41.780 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 403726925826 2026-02-17T11:54:41.780 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:54:41.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 403726925826 -lt 403726925826 2026-02-17T11:54:41.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:41.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-373662154757 2026-02-17T11:54:41.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:41.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:54:41.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-373662154757 2026-02-17T11:54:41.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:41.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=373662154757 2026-02-17T11:54:41.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 373662154757' 2026-02-17T11:54:41.953 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 373662154757 2026-02-17T11:54:41.954 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:54:42.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 373662154756 -lt 373662154757 2026-02-17T11:54:42.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:54:43.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:54:43.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:54:43.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 373662154756 -lt 373662154757 2026-02-17T11:54:43.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:54:44.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:54:44.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:54:44.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 373662154757 -lt 373662154757 2026-02-17T11:54:44.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:44.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-292057776142 2026-02-17T11:54:44.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:44.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:54:44.429 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-292057776142 2026-02-17T11:54:44.429 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:44.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776142 2026-02-17T11:54:44.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 292057776142' 2026-02-17T11:54:44.431 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 292057776142 2026-02-17T11:54:44.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:54:44.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776142 -lt 292057776142 2026-02-17T11:54:44.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:54:44.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:54:44.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:54:44.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:54:44.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:54:44.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:54:44.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:54:44.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:54:44.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:54:44.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:54:44.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:54:44.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:54:44.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:54:44.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:54:44.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:54:45.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:54:45.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:54:45.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:54:45.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3669: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-17T11:54:45.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:54:45.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:54:45.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:54:45.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:54:45.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-17T11:54:45.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:54:45.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:54:45.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:54:45.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:54:45.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:54:45.130 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:54:45.130 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:54:45.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:54:45.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:54:45.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:54:46.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:54:46.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-17T11:54:46.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:54:46.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:54:46.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:54:46.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:54:46.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:54:46.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-17T11:54:47.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:54:47.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:54:47.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:54:47.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:54:47.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:54:47.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:54:47.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:54:47.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:54:47.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:54:47.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:54:47.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:54:47.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:54:47.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:54:47.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:54:47.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:54:47.604 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:54:47.604 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:54:47.604 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:54:47.606 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:54:47.606 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:54:47.606 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:54:47.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:54:47.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:54:47.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:54:47.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:54:47.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:54:47.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:54:47.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:54:47.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:54:47.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:54:47.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:54:47.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:54:47.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:54:47.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:54:47.608 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:54:47.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:54:47.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:54:47.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:54:47.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:54:47.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:54:47.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:54:47.628 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:47.621+0000 7fdbb47f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:54:47.630 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:47.621+0000 7fdbb47f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:54:47.631 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:47.621+0000 7fdbb47f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:54:47.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:54:47.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:54:47.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:54:47.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:54:47.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:54:47.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:47.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:54:47.762 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:54:47.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:47.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:54:47.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:48.874 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:48.865+0000 7fdbb47f38c0 -1 Falling back to public interface 2026-02-17T11:54:48.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:48.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:48.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:54:48.913 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:54:48.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:48.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:54:49.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:50.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:50.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:50.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:54:50.073 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:54:50.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:50.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:54:50.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:51.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:51.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:51.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:54:51.228 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:54:51.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:51.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:54:51.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:54:51.839 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:54:51.829+0000 7fdbb47f38c0 -1 osd.0 95 log_to_monitors true 2026-02-17T11:54:52.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:54:52.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:54:52.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:54:52.380 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:54:52.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:54:52.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:54:52.530 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 99 up_thru 99 down_at 96 last_clean_interval [94,95) [v2:127.0.0.1:6802/3213013432,v1:127.0.0.1:6803/3213013432] [v2:127.0.0.1:6804/3213013432,v1:127.0.0.1:6805/3213013432] exists,up b7951352-7c09-4b75-abda-71432d2b706a 2026-02-17T11:54:52.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:54:52.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:54:52.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:54:52.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:54:52.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:54:52.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:54:52.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:54:52.531 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:54:52.532 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:54:52.532 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:54:52.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:54:52.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:54:52.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:54:52.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:54:52.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:54:52.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:54:52.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:54:52.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:54:52.646 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:54:52.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:54:52.801 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:54:52.801 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:54:52.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:54:52.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:52.801 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:54:52.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=425201762306 2026-02-17T11:54:52.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 425201762306 2026-02-17T11:54:52.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762306' 2026-02-17T11:54:52.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:52.875 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:54:52.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=373662154760 2026-02-17T11:54:52.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 373662154760 2026-02-17T11:54:52.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762306 1-373662154760' 2026-02-17T11:54:52.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:52.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:54:53.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776145 2026-02-17T11:54:53.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776145 2026-02-17T11:54:53.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762306 1-373662154760 2-292057776145' 2026-02-17T11:54:53.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:53.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-425201762306 2026-02-17T11:54:53.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:53.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:54:53.021 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-425201762306 2026-02-17T11:54:53.021 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:53.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=425201762306 2026-02-17T11:54:53.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 425201762306' 2026-02-17T11:54:53.023 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 425201762306 2026-02-17T11:54:53.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:54:53.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 403726925826 -lt 425201762306 2026-02-17T11:54:53.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:54:54.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:54:54.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:54:54.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762306 -lt 425201762306 2026-02-17T11:54:54.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:54.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-373662154760 2026-02-17T11:54:54.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:54.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:54:54.347 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-373662154760 2026-02-17T11:54:54.347 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:54.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=373662154760 2026-02-17T11:54:54.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 373662154760' 2026-02-17T11:54:54.349 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 373662154760 2026-02-17T11:54:54.350 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:54:54.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 373662154760 -lt 373662154760 2026-02-17T11:54:54.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:54.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-292057776145 2026-02-17T11:54:54.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:54.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:54:54.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-292057776145 2026-02-17T11:54:54.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:54.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776145 2026-02-17T11:54:54.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 292057776145' 2026-02-17T11:54:54.506 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 292057776145 2026-02-17T11:54:54.507 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:54:54.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776145 -lt 292057776145 2026-02-17T11:54:54.667 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:54:54.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:54:54.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:54:54.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:54:54.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:54:54.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:54:54.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:54:54.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:54:54.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:54:54.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:54:54.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:54:55.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:54:55.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:54:55.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:54:55.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:54:55.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:54:55.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:54:55.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:54:55.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3670: corrupt_scrub_erasure: rm -f td/osd-scrub-repair/hinfo 2026-02-17T11:54:55.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3676: corrupt_scrub_erasure: get_pg ecpool EOBJ0 2026-02-17T11:54:55.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:54:55.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=EOBJ0 2026-02-17T11:54:55.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool EOBJ0 2026-02-17T11:54:55.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:54:55.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3676: corrupt_scrub_erasure: local pg=3.0 2026-02-17T11:54:55.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3678: corrupt_scrub_erasure: pg_scrub 3.0 2026-02-17T11:54:55.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=3.0 2026-02-17T11:54:55.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 3.0 2026-02-17T11:54:55.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=3.0 2026-02-17T11:54:55.349 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T11:54:55.350 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:54:55.350 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:54:55.350 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:54:55.350 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:54:55.350 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:54:55.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T11:54:55.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T11:54:55.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T11:54:55.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T11:54:55.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:54:55.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:54:55.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:54:55.787 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:54:55.787 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:54:55.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:54:55.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:55.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:54:55.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=425201762307 2026-02-17T11:54:55.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 425201762307 2026-02-17T11:54:55.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762307' 2026-02-17T11:54:55.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:55.862 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:54:55.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=373662154762 2026-02-17T11:54:55.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 373662154762 2026-02-17T11:54:55.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762307 1-373662154762' 2026-02-17T11:54:55.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:54:55.934 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:54:56.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776147 2026-02-17T11:54:56.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776147 2026-02-17T11:54:56.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762307 1-373662154762 2-292057776147' 2026-02-17T11:54:56.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:56.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-425201762307 2026-02-17T11:54:56.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:56.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:54:56.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-425201762307 2026-02-17T11:54:56.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:56.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=425201762307 2026-02-17T11:54:56.013 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 425201762307 2026-02-17T11:54:56.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 425201762307' 2026-02-17T11:54:56.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:54:56.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762306 -lt 425201762307 2026-02-17T11:54:56.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:54:57.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:54:57.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:54:57.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762306 -lt 425201762307 2026-02-17T11:54:57.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:54:58.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:54:58.330 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:54:58.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762308 -lt 425201762307 2026-02-17T11:54:58.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:58.483 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-373662154762 2026-02-17T11:54:58.483 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:58.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:54:58.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-373662154762 2026-02-17T11:54:58.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:58.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=373662154762 2026-02-17T11:54:58.487 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 373662154762 2026-02-17T11:54:58.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 373662154762' 2026-02-17T11:54:58.487 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:54:58.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 373662154762 -lt 373662154762 2026-02-17T11:54:58.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:54:58.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-292057776147 2026-02-17T11:54:58.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:54:58.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:54:58.653 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-292057776147 2026-02-17T11:54:58.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:54:58.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776147 2026-02-17T11:54:58.655 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 292057776147 2026-02-17T11:54:58.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 292057776147' 2026-02-17T11:54:58.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:54:58.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776147 -lt 292057776147 2026-02-17T11:54:58.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T11:54:58.816 INFO:tasks.workunit.client.0.trial128.stdout:#---------- 3.0 loop 0 2026-02-17T11:54:58.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-17T11:54:58.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 3.0 2026-02-17T11:54:58.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=3.0 2026-02-17T11:54:58.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T11:54:58.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 3.0 query 2026-02-17T11:54:58.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T11:54:58.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T11:54:58.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T11:54:58.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T11:54:58.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T11:54:58.888 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 3.0 2026-02-17T11:54:58.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:54:58.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:54:58.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:54:58.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T11:54:59.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-17T11:52:53.026952+0000 2026-02-17T11:54:59.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 3.0 2026-02-17T11:54:59.183 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 3.0s0 on osd.1 to scrub 2026-02-17T11:54:59.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 3.0 2026-02-17T11:52:53.026952+0000 2026-02-17T11:54:59.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T11:54:59.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:52:53.026952+0000 2026-02-17T11:54:59.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:54:59.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:54:59.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:54:59.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T11:54:59.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:54:59.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:54:59.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:54:59.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T11:54:59.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:52:53.026952+0000 '>' 2026-02-17T11:52:53.026952+0000 2026-02-17T11:54:59.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:55:00.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:55:00.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:55:00.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T11:55:00.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:55:00.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:55:00.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:55:00.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T11:55:00.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:52:53.026952+0000 '>' 2026-02-17T11:52:53.026952+0000 2026-02-17T11:55:00.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:55:01.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:55:01.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:55:01.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T11:55:01.497 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:55:01.497 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:55:01.497 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:55:01.497 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T11:55:01.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:52:53.026952+0000 '>' 2026-02-17T11:52:53.026952+0000 2026-02-17T11:55:01.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:55:02.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:55:02.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:55:02.649 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T11:55:02.649 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:55:02.649 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:55:02.649 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:55:02.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T11:55:02.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:54:59.401019+0000 '>' 2026-02-17T11:52:53.026952+0000 2026-02-17T11:55:02.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:55:02.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3680: corrupt_scrub_erasure: rados list-inconsistent-pg ecpool 2026-02-17T11:55:02.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3682: corrupt_scrub_erasure: jq '. | length' td/osd-scrub-repair/json 2026-02-17T11:55:02.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3682: corrupt_scrub_erasure: test 1 = 1 2026-02-17T11:55:02.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3684: corrupt_scrub_erasure: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-17T11:55:02.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3684: corrupt_scrub_erasure: test 3.0 = 3.0 2026-02-17T11:55:02.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3686: corrupt_scrub_erasure: rados list-inconsistent-obj 3.0 2026-02-17T11:55:02.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3688: corrupt_scrub_erasure: jq .epoch td/osd-scrub-repair/json 2026-02-17T11:55:02.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3688: corrupt_scrub_erasure: epoch=99 2026-02-17T11:55:02.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3690: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-17T11:55:02.869 INFO:tasks.workunit.client.0.trial128.stderr: . as $in 2026-02-17T11:55:02.870 INFO:tasks.workunit.client.0.trial128.stderr: | if type == "object" then 2026-02-17T11:55:02.870 INFO:tasks.workunit.client.0.trial128.stderr: reduce keys[] as $key 2026-02-17T11:55:02.870 INFO:tasks.workunit.client.0.trial128.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T11:55:02.870 INFO:tasks.workunit.client.0.trial128.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T11:55:02.870 INFO:tasks.workunit.client.0.trial128.stderr: else f 2026-02-17T11:55:02.870 INFO:tasks.workunit.client.0.trial128.stderr: end; 2026-02-17T11:55:02.870 INFO:tasks.workunit.client.0.trial128.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T11:55:02.870 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T11:55:02.870 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T11:55:02.870 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T11:55:02.870 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-17T11:55:02.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3690: corrupt_scrub_erasure: jq .inconsistents 2026-02-17T11:55:02.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3690: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T11:55:02.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4310: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-17T11:55:02.888 INFO:tasks.workunit.client.0.trial128.stderr: . as $in 2026-02-17T11:55:02.888 INFO:tasks.workunit.client.0.trial128.stderr: | if type == "object" then 2026-02-17T11:55:02.888 INFO:tasks.workunit.client.0.trial128.stderr: reduce keys[] as $key 2026-02-17T11:55:02.888 INFO:tasks.workunit.client.0.trial128.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T11:55:02.888 INFO:tasks.workunit.client.0.trial128.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T11:55:02.888 INFO:tasks.workunit.client.0.trial128.stderr: else f 2026-02-17T11:55:02.888 INFO:tasks.workunit.client.0.trial128.stderr: end; 2026-02-17T11:55:02.889 INFO:tasks.workunit.client.0.trial128.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T11:55:02.889 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T11:55:02.889 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T11:55:02.889 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T11:55:02.889 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-17T11:55:02.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4310: corrupt_scrub_erasure: jq .inconsistents 2026-02-17T11:55:02.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4310: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T11:55:02.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4311: corrupt_scrub_erasure: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T11:55:02.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T11:55:02.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4312: corrupt_scrub_erasure: test no = yes 2026-02-17T11:55:02.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4317: corrupt_scrub_erasure: test '' = yes 2026-02-17T11:55:02.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4322: corrupt_scrub_erasure: pg_deep_scrub 3.0 2026-02-17T11:55:02.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: pg_deep_scrub: local pgid=3.0 2026-02-17T11:55:02.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: pg_deep_scrub: wait_for_pg_clean 3.0 2026-02-17T11:55:02.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=3.0 2026-02-17T11:55:02.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T11:55:02.915 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:55:02.915 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:55:02.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:55:02.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:55:02.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:55:03.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T11:55:03.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T11:55:03.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T11:55:03.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T11:55:03.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:55:03.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:55:03.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:55:03.359 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:55:03.359 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:55:03.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:55:03.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:55:03.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:55:03.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=425201762310 2026-02-17T11:55:03.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 425201762310 2026-02-17T11:55:03.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762310' 2026-02-17T11:55:03.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:55:03.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:55:03.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=373662154764 2026-02-17T11:55:03.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 373662154764 2026-02-17T11:55:03.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762310 1-373662154764' 2026-02-17T11:55:03.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:55:03.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:55:03.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=292057776149 2026-02-17T11:55:03.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 292057776149 2026-02-17T11:55:03.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-425201762310 1-373662154764 2-292057776149' 2026-02-17T11:55:03.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:55:03.574 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-425201762310 2026-02-17T11:55:03.574 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:55:03.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:55:03.576 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-425201762310 2026-02-17T11:55:03.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:55:03.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=425201762310 2026-02-17T11:55:03.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 425201762310' 2026-02-17T11:55:03.578 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 425201762310 2026-02-17T11:55:03.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:55:03.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762308 -lt 425201762310 2026-02-17T11:55:03.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:55:04.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:55:04.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:55:04.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762310 -lt 425201762310 2026-02-17T11:55:04.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:55:04.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-373662154764 2026-02-17T11:55:04.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:55:04.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:55:04.902 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-373662154764 2026-02-17T11:55:04.902 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:55:04.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=373662154764 2026-02-17T11:55:04.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 373662154764' 2026-02-17T11:55:04.904 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 373662154764 2026-02-17T11:55:04.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:55:05.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 373662154764 -lt 373662154764 2026-02-17T11:55:05.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:55:05.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-292057776149 2026-02-17T11:55:05.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:55:05.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:55:05.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-292057776149 2026-02-17T11:55:05.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:55:05.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=292057776149 2026-02-17T11:55:05.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 292057776149' 2026-02-17T11:55:05.071 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 292057776149 2026-02-17T11:55:05.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:55:05.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 292057776149 -lt 292057776149 2026-02-17T11:55:05.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T11:55:05.236 INFO:tasks.workunit.client.0.trial128.stdout:#---------- 3.0 loop 0 2026-02-17T11:55:05.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-17T11:55:05.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 3.0 2026-02-17T11:55:05.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=3.0 2026-02-17T11:55:05.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T11:55:05.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 3.0 query 2026-02-17T11:55:05.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T11:55:05.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean+inconsistent 2026-02-17T11:55:05.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean+inconsistent == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T11:55:05.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T11:55:05.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T11:55:05.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:55:05.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:55:05.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:55:05.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:55:05.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:55:05.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: local last_scrub=2026-02-17T11:52:53.026952+0000 2026-02-17T11:55:05.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: pg_deep_scrub: ceph pg deep-scrub 3.0 2026-02-17T11:55:05.604 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 3.0s0 on osd.1 to deep-scrub 2026-02-17T11:55:05.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: pg_deep_scrub: wait_for_scrub 3.0 2026-02-17T11:52:53.026952+0000 last_deep_scrub_stamp 2026-02-17T11:55:05.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T11:55:05.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:52:53.026952+0000 2026-02-17T11:55:05.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-17T11:55:05.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:55:05.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:55:05.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:55:05.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:55:05.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:55:05.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:55:05.614 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:55:05.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:52:53.026952+0000 '>' 2026-02-17T11:52:53.026952+0000 2026-02-17T11:55:05.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:55:06.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:55:06.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:55:06.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:55:06.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:55:06.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:55:06.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:55:06.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:55:06.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:52:53.026952+0000 '>' 2026-02-17T11:52:53.026952+0000 2026-02-17T11:55:06.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:55:07.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:55:07.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:55:07.931 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:55:07.931 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:55:07.931 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:55:07.932 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:55:07.932 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:55:08.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:52:53.026952+0000 '>' 2026-02-17T11:52:53.026952+0000 2026-02-17T11:55:08.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:55:09.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:55:09.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:55:09.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:55:09.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:55:09.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:55:09.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:55:09.089 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:55:09.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:52:53.026952+0000 '>' 2026-02-17T11:52:53.026952+0000 2026-02-17T11:55:09.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:55:10.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:55:10.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:55:10.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:55:10.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:55:10.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:55:10.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:55:10.245 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:55:10.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:52:53.026952+0000 '>' 2026-02-17T11:52:53.026952+0000 2026-02-17T11:55:10.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:55:11.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:55:11.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:55:11.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:55:11.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:55:11.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:55:11.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:55:11.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:55:11.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:52:53.026952+0000 '>' 2026-02-17T11:52:53.026952+0000 2026-02-17T11:55:11.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:55:12.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:55:12.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:55:12.541 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:55:12.541 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:55:12.541 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:55:12.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:55:12.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:55:12.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:55:06.306780+0000 '>' 2026-02-17T11:52:53.026952+0000 2026-02-17T11:55:12.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:55:12.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4324: corrupt_scrub_erasure: rados list-inconsistent-pg ecpool 2026-02-17T11:55:12.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4326: corrupt_scrub_erasure: jq '. | length' td/osd-scrub-repair/json 2026-02-17T11:55:12.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4326: corrupt_scrub_erasure: test 1 = 1 2026-02-17T11:55:12.722 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4328: corrupt_scrub_erasure: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-17T11:55:12.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4328: corrupt_scrub_erasure: test 3.0 = 3.0 2026-02-17T11:55:12.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4330: corrupt_scrub_erasure: rados list-inconsistent-obj 3.0 2026-02-17T11:55:12.749 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4332: corrupt_scrub_erasure: jq .epoch td/osd-scrub-repair/json 2026-02-17T11:55:12.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4332: corrupt_scrub_erasure: epoch=99 2026-02-17T11:55:12.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4334: corrupt_scrub_erasure: '[' false = true ']' 2026-02-17T11:55:12.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4991: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-17T11:55:12.762 INFO:tasks.workunit.client.0.trial128.stderr: . as $in 2026-02-17T11:55:12.762 INFO:tasks.workunit.client.0.trial128.stderr: | if type == "object" then 2026-02-17T11:55:12.762 INFO:tasks.workunit.client.0.trial128.stderr: reduce keys[] as $key 2026-02-17T11:55:12.762 INFO:tasks.workunit.client.0.trial128.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T11:55:12.762 INFO:tasks.workunit.client.0.trial128.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T11:55:12.763 INFO:tasks.workunit.client.0.trial128.stderr: else f 2026-02-17T11:55:12.763 INFO:tasks.workunit.client.0.trial128.stderr: end; 2026-02-17T11:55:12.763 INFO:tasks.workunit.client.0.trial128.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T11:55:12.763 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T11:55:12.763 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T11:55:12.763 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T11:55:12.763 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-17T11:55:12.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4991: corrupt_scrub_erasure: jq .inconsistents 2026-02-17T11:55:12.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4991: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T11:55:12.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5724: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-17T11:55:12.780 INFO:tasks.workunit.client.0.trial128.stderr: . as $in 2026-02-17T11:55:12.780 INFO:tasks.workunit.client.0.trial128.stderr: | if type == "object" then 2026-02-17T11:55:12.780 INFO:tasks.workunit.client.0.trial128.stderr: reduce keys[] as $key 2026-02-17T11:55:12.780 INFO:tasks.workunit.client.0.trial128.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T11:55:12.780 INFO:tasks.workunit.client.0.trial128.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T11:55:12.780 INFO:tasks.workunit.client.0.trial128.stderr: else f 2026-02-17T11:55:12.780 INFO:tasks.workunit.client.0.trial128.stderr: end; 2026-02-17T11:55:12.780 INFO:tasks.workunit.client.0.trial128.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T11:55:12.780 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T11:55:12.781 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T11:55:12.781 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T11:55:12.781 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-17T11:55:12.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5724: corrupt_scrub_erasure: jq .inconsistents 2026-02-17T11:55:12.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5724: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T11:55:12.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5725: corrupt_scrub_erasure: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T11:55:12.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T11:55:12.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5726: corrupt_scrub_erasure: test no = yes 2026-02-17T11:55:12.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5737: corrupt_scrub_erasure: test '' = yes 2026-02-17T11:55:12.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5742: corrupt_scrub_erasure: ceph osd pool rm ecpool ecpool --yes-i-really-really-mean-it 2026-02-17T11:55:13.002 INFO:tasks.workunit.client.0.trial128.stderr:pool 'ecpool' removed 2026-02-17T11:55:13.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T11:55:13.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:55:13.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:55:13.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:55:13.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:55:13.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:55:13.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:55:13.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:55:13.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:55:13.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:55:13.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:55:13.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:55:13.127 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:55:13.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:55:13.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:55:13.128 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:55:13.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:55:13.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:55:13.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:55:13.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:55:13.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:55:13.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:55:13.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:55:13.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:55:13.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:55:13.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:55:13.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:55:13.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:55:13.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:55:13.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T11:55:13.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T11:55:13.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T11:55:13.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T11:55:13.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:55:13.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:55:13.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:55:13.163 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:55:13.164 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:55:13.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:55:13.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:55:13.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:55:13.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:55:13.166 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:55:13.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:55:13.167 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:55:13.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:55:13.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:55:13.169 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:55:13.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:55:13.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:55:13.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:55:13.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:55:13.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:55:13.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:55:13.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:55:13.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:55:13.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:55:13.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:55:13.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:55:13.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:55:13.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:55:13.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T11:55:13.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:55:13.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:55:13.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:55:13.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T11:55:13.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:55:13.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T11:55:13.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:55:13.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T11:55:13.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_scrub_erasure_overwrites td/osd-scrub-repair 2026-02-17T11:55:13.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5750: TEST_corrupt_scrub_erasure_overwrites: '[' true = true ']' 2026-02-17T11:55:13.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5751: TEST_corrupt_scrub_erasure_overwrites: corrupt_scrub_erasure td/osd-scrub-repair true 2026-02-17T11:55:13.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3596: corrupt_scrub_erasure: local dir=td/osd-scrub-repair 2026-02-17T11:55:13.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3597: corrupt_scrub_erasure: local allow_overwrites=true 2026-02-17T11:55:13.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3598: corrupt_scrub_erasure: local poolname=ecpool 2026-02-17T11:55:13.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3599: corrupt_scrub_erasure: local total_objs=7 2026-02-17T11:55:13.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3601: corrupt_scrub_erasure: run_mon td/osd-scrub-repair a 2026-02-17T11:55:13.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T11:55:13.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T11:55:13.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T11:55:13.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T11:55:13.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T11:55:13.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T11:55:13.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T11:55:13.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:55:13.203 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:55:13.203 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:55:13.203 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:55:13.203 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:55:13.203 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:55:13.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T11:55:13.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T11:55:13.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T11:55:13.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:55:13.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:55:13.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T11:55:13.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T11:55:13.234 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:55:13.234 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:55:13.234 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:55:13.234 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:55:13.234 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:55:13.234 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:55:13.235 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:55:13.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:55:13.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T11:55:13.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T11:55:13.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:55:13.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:55:13.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T11:55:13.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T11:55:13.292 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:55:13.292 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:55:13.292 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:55:13.293 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:55:13.293 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:55:13.293 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:55:13.293 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:55:13.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:55:13.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T11:55:13.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3602: corrupt_scrub_erasure: run_mgr td/osd-scrub-repair x 2026-02-17T11:55:13.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T11:55:13.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T11:55:13.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T11:55:13.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T11:55:13.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T11:55:13.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T11:55:13.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T11:55:13.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:55:13.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:55:13.441 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:55:13.441 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:55:13.441 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:55:13.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:55:13.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:55:13.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:55:13.461 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3603: corrupt_scrub_erasure: seq 0 2 2026-02-17T11:55:13.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3603: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-17T11:55:13.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3604: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 0 2026-02-17T11:55:13.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:55:13.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:55:13.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T11:55:13.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:55:13.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:55:13.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:55:13.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:55:13.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:55:13.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:55:13.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:55:13.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:55:13.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:55:13.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:55:13.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:55:13.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:55:13.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:55:13.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:55:13.466 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:55:13.466 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:55:13.466 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:55:13.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:55:13.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:55:13.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:55:13.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:55:13.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:55:13.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:55:13.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:55:13.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:55:13.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:55:13.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:55:13.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:55:13.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:55:13.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:55:13.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:55:13.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:55:13.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=9dcfb325-6c1f-4246-a9b2-e39bf506f0d4 2026-02-17T11:55:13.469 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 9dcfb325-6c1f-4246-a9b2-e39bf506f0d4 2026-02-17T11:55:13.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 9dcfb325-6c1f-4246-a9b2-e39bf506f0d4' 2026-02-17T11:55:13.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:55:13.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAhV5Rp0DINHBAADUVJtooIumQHbdvIyMX/fg== 2026-02-17T11:55:13.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAhV5Rp0DINHBAADUVJtooIumQHbdvIyMX/fg=="}' 2026-02-17T11:55:13.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 9dcfb325-6c1f-4246-a9b2-e39bf506f0d4 -i td/osd-scrub-repair/0/new.json 2026-02-17T11:55:13.569 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:55:13.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T11:55:13.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAhV5Rp0DINHBAADUVJtooIumQHbdvIyMX/fg== --osd-uuid 9dcfb325-6c1f-4246-a9b2-e39bf506f0d4 2026-02-17T11:55:13.597 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:13.589+0000 7f69dd12f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:13.599 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:13.589+0000 7f69dd12f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:13.600 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:13.589+0000 7f69dd12f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:13.600 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:13.593+0000 7f69dd12f8c0 -1 bdev(0x55ed88101c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:55:13.600 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:13.593+0000 7f69dd12f8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T11:55:16.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T11:55:16.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:55:16.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T11:55:16.266 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:55:16.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:55:16.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T11:55:16.457 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:55:16.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:55:16.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:55:16.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:55:16.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:55:16.479 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:16.469+0000 7fd07ef378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:16.481 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:16.473+0000 7fd07ef378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:16.482 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:16.473+0000 7fd07ef378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:16.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T11:55:16.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:55:16.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:55:16.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:55:16.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:55:16.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:16.616 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:55:16.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:55:16.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:16.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:55:16.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:17.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:17.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:17.770 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:55:17.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:55:17.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:17.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:55:17.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:17.970 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:17.960+0000 7fd07ef378c0 -1 Falling back to public interface 2026-02-17T11:55:18.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:18.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:18.918 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:55:18.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:55:18.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:18.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:55:19.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:20.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:20.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:20.069 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:55:20.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:55:20.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:20.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:55:20.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:20.962 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:20.952+0000 7fd07ef378c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:55:21.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:21.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:21.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:55:21.220 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:55:21.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:21.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:55:21.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:22.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:22.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:22.372 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:55:22.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:55:22.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:22.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:55:22.520 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/9066239,v1:127.0.0.1:6803/9066239] [v2:127.0.0.1:6804/9066239,v1:127.0.0.1:6805/9066239] exists,up 9dcfb325-6c1f-4246-a9b2-e39bf506f0d4 2026-02-17T11:55:22.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:55:22.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:55:22.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:55:22.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3603: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-17T11:55:22.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3604: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 1 2026-02-17T11:55:22.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:55:22.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:55:22.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T11:55:22.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:55:22.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:55:22.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:55:22.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:55:22.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:55:22.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:55:22.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:55:22.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:55:22.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:55:22.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:55:22.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:55:22.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:55:22.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:55:22.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:55:22.525 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:55:22.525 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:55:22.525 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:55:22.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:55:22.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:55:22.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:55:22.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:55:22.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:55:22.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:55:22.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:55:22.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:55:22.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:55:22.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:55:22.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:55:22.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:55:22.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:55:22.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:55:22.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:55:22.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=77e8fc16-8961-4fc8-a2d2-b3df194c9c30 2026-02-17T11:55:22.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 77e8fc16-8961-4fc8-a2d2-b3df194c9c30' 2026-02-17T11:55:22.528 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 77e8fc16-8961-4fc8-a2d2-b3df194c9c30 2026-02-17T11:55:22.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:55:22.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAqV5RpJJeuHxAAkt4cpR/hL727u81ZOUDFVg== 2026-02-17T11:55:22.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAqV5RpJJeuHxAAkt4cpR/hL727u81ZOUDFVg=="}' 2026-02-17T11:55:22.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 77e8fc16-8961-4fc8-a2d2-b3df194c9c30 -i td/osd-scrub-repair/1/new.json 2026-02-17T11:55:22.685 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:55:22.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T11:55:22.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAqV5RpJJeuHxAAkt4cpR/hL727u81ZOUDFVg== --osd-uuid 77e8fc16-8961-4fc8-a2d2-b3df194c9c30 2026-02-17T11:55:22.712 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:22.700+0000 7fbbb84fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:22.713 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:22.704+0000 7fbbb84fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:22.714 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:22.704+0000 7fbbb84fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:22.715 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:22.704+0000 7fbbb84fc8c0 -1 bdev(0x5581c957bc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:55:22.715 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:22.704+0000 7fbbb84fc8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T11:55:24.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T11:55:24.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:55:24.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T11:55:24.974 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:55:24.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:55:25.186 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:55:25.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T11:55:25.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:55:25.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:55:25.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:55:25.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:55:25.201 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:25.192+0000 7f03b100e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:25.203 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:25.192+0000 7f03b100e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:25.204 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:25.196+0000 7f03b100e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:25.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T11:55:25.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:55:25.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:55:25.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:55:25.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:55:25.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:25.343 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:55:25.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:55:25.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:25.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:55:25.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:26.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:26.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:26.493 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:55:26.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:55:26.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:26.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:55:26.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:26.710 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:26.700+0000 7f03b100e8c0 -1 Falling back to public interface 2026-02-17T11:55:27.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:27.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:27.645 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:55:27.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:55:27.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:27.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:55:27.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:28.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:28.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:28.796 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:55:28.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:55:28.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:28.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:55:28.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:29.699 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:29.688+0000 7f03b100e8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:55:29.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:29.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:29.949 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:55:29.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:55:29.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:29.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:55:30.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:31.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:31.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:31.108 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:55:31.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:55:31.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:31.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:55:31.258 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3031521137,v1:127.0.0.1:6811/3031521137] [v2:127.0.0.1:6812/3031521137,v1:127.0.0.1:6813/3031521137] exists,up 77e8fc16-8961-4fc8-a2d2-b3df194c9c30 2026-02-17T11:55:31.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:55:31.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:55:31.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:55:31.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3603: corrupt_scrub_erasure: for id in $(seq 0 2) 2026-02-17T11:55:31.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3604: corrupt_scrub_erasure: run_osd td/osd-scrub-repair 2 2026-02-17T11:55:31.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:55:31.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:55:31.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T11:55:31.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:55:31.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:55:31.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:55:31.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:55:31.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:55:31.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:55:31.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:55:31.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:55:31.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:55:31.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:55:31.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:55:31.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:55:31.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:55:31.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:55:31.263 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:55:31.263 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:55:31.264 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:55:31.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:55:31.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:55:31.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:55:31.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:55:31.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:55:31.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:55:31.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:55:31.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:55:31.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:55:31.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:55:31.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:55:31.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:55:31.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:55:31.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:55:31.267 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:55:31.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=218a6832-4f40-4272-a819-3caf6dbea61c 2026-02-17T11:55:31.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 218a6832-4f40-4272-a819-3caf6dbea61c' 2026-02-17T11:55:31.268 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 218a6832-4f40-4272-a819-3caf6dbea61c 2026-02-17T11:55:31.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:55:31.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAzV5RppxX8DxAAIeSyd+f0PjcxfZSLXq8eDQ== 2026-02-17T11:55:31.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAzV5RppxX8DxAAIeSyd+f0PjcxfZSLXq8eDQ=="}' 2026-02-17T11:55:31.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 218a6832-4f40-4272-a819-3caf6dbea61c -i td/osd-scrub-repair/2/new.json 2026-02-17T11:55:31.431 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:55:31.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T11:55:31.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAzV5RppxX8DxAAIeSyd+f0PjcxfZSLXq8eDQ== --osd-uuid 218a6832-4f40-4272-a819-3caf6dbea61c 2026-02-17T11:55:31.458 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:31.448+0000 7f3c7b7de8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:31.459 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:31.448+0000 7f3c7b7de8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:31.460 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:31.452+0000 7f3c7b7de8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:31.461 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:31.452+0000 7f3c7b7de8c0 -1 bdev(0x55ce59bcdc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T11:55:31.461 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:31.452+0000 7f3c7b7de8c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T11:55:33.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T11:55:33.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:55:33.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T11:55:33.749 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T11:55:33.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:55:33.939 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T11:55:33.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T11:55:33.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:55:33.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:55:33.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:55:33.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:55:33.962 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:33.952+0000 7f73999648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:33.964 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:33.952+0000 7f73999648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:33.965 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:33.956+0000 7f73999648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:34.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T11:55:34.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:55:34.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:55:34.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:55:34.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:55:34.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:34.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:55:34.094 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:55:34.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:34.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:55:34.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:35.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:35.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:35.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:55:35.247 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:55:35.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:35.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:55:35.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:35.461 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:35.452+0000 7f73999648c0 -1 Falling back to public interface 2026-02-17T11:55:36.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:36.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:36.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:55:36.397 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:55:36.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:36.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:55:36.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:37.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:37.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:37.556 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:55:37.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:55:37.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:37.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:55:37.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:38.425 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:38.416+0000 7f73999648c0 -1 osd.2 0 log_to_monitors true 2026-02-17T11:55:38.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:38.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:38.710 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:55:38.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:55:38.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:38.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:55:38.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:39.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:39.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:39.863 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:55:39.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:55:39.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:39.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:55:40.014 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1154434444,v1:127.0.0.1:6819/1154434444] [v2:127.0.0.1:6820/1154434444,v1:127.0.0.1:6821/1154434444] exists,up 218a6832-4f40-4272-a819-3caf6dbea61c 2026-02-17T11:55:40.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:55:40.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:55:40.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:55:40.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3606: corrupt_scrub_erasure: create_rbd_pool 2026-02-17T11:55:40.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T11:55:40.156 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' does not exist 2026-02-17T11:55:40.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T11:55:40.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T11:55:40.378 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T11:55:40.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:55:41.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T11:55:41.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3607: corrupt_scrub_erasure: create_pool foo 1 2026-02-17T11:55:41.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 1 2026-02-17T11:55:41.870 INFO:tasks.workunit.client.0.trial128.stderr:pool 'foo' created 2026-02-17T11:55:41.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:55:42.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3609: corrupt_scrub_erasure: create_ec_pool ecpool true k=2 m=1 stripe_unit=2K --force 2026-02-17T11:55:42.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T11:55:42.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T11:55:42.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=true 2026-02-17T11:55:42.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T11:55:42.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 stripe_unit=2K --force 2026-02-17T11:55:43.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T11:55:43.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T11:55:43.423 INFO:tasks.workunit.client.0.trial128.stderr:pool 'ecpool' created 2026-02-17T11:55:43.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:55:44.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' true = true ']' 2026-02-17T11:55:44.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2508: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-17T11:55:44.582 INFO:tasks.workunit.client.0.trial128.stderr:set pool 3 allow_ec_overwrites to true 2026-02-17T11:55:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T11:55:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:55:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:55:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:55:44.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:55:44.594 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:55:44.594 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:55:44.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:55:44.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:55:44.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:55:44.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:55:44.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:55:44.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:55:44.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:55:44.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:55:44.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:55:44.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:55:44.853 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:55:44.853 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:55:44.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:55:44.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:55:44.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:55:44.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836486 2026-02-17T11:55:44.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836486 2026-02-17T11:55:44.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486' 2026-02-17T11:55:44.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:55:44.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:55:44.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T11:55:44.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T11:55:44.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-42949672964' 2026-02-17T11:55:44.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:55:44.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:55:45.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542147 2026-02-17T11:55:45.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542147 2026-02-17T11:55:45.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-42949672964 2-60129542147' 2026-02-17T11:55:45.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:55:45.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836486 2026-02-17T11:55:45.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:55:45.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:55:45.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836486 2026-02-17T11:55:45.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:55:45.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836486 2026-02-17T11:55:45.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-17T11:55:45.075 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836486 2026-02-17T11:55:45.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:55:45.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-17T11:55:45.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:55:46.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:55:46.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:55:46.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-17T11:55:46.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:55:46.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T11:55:46.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:55:46.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:55:46.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T11:55:46.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:55:46.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T11:55:46.393 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672964 2026-02-17T11:55:46.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T11:55:46.393 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:55:46.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T11:55:46.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:55:46.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542147 2026-02-17T11:55:46.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:55:46.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:55:46.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542147 2026-02-17T11:55:46.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:55:46.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542147 2026-02-17T11:55:46.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-17T11:55:46.546 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542147 2026-02-17T11:55:46.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:55:46.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-17T11:55:46.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:55:46.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:55:46.703 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:55:46.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:55:46.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:55:46.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:55:46.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:55:46.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:55:46.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:55:46.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:55:46.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:55:47.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:55:47.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:55:47.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:55:47.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:55:47.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:55:47.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:55:47.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:55:47.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T11:55:47.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3610: corrupt_scrub_erasure: wait_for_clean 2026-02-17T11:55:47.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:55:47.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:55:47.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:55:47.205 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:55:47.205 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:55:47.205 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:55:47.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:55:47.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:55:47.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:55:47.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:55:47.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:55:47.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:55:47.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:55:47.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:55:47.316 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:55:47.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:55:47.474 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:55:47.475 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:55:47.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:55:47.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:55:47.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:55:47.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836488 2026-02-17T11:55:47.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836488 2026-02-17T11:55:47.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488' 2026-02-17T11:55:47.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:55:47.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:55:47.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672966 2026-02-17T11:55:47.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672966 2026-02-17T11:55:47.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-42949672966' 2026-02-17T11:55:47.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:55:47.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:55:47.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542148 2026-02-17T11:55:47.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542148 2026-02-17T11:55:47.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-42949672966 2-60129542148' 2026-02-17T11:55:47.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:55:47.694 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836488 2026-02-17T11:55:47.694 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:55:47.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:55:47.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836488 2026-02-17T11:55:47.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:55:47.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836488 2026-02-17T11:55:47.698 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836488 2026-02-17T11:55:47.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-17T11:55:47.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:55:47.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-17T11:55:47.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:55:47.864 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672966 2026-02-17T11:55:47.864 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:55:47.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:55:47.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672966 2026-02-17T11:55:47.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:55:47.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672966 2026-02-17T11:55:47.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2026-02-17T11:55:47.868 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672966 2026-02-17T11:55:47.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:55:48.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672966 -lt 42949672966 2026-02-17T11:55:48.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:55:48.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542148 2026-02-17T11:55:48.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:55:48.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:55:48.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542148 2026-02-17T11:55:48.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:55:48.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542148 2026-02-17T11:55:48.032 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542148 2026-02-17T11:55:48.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-17T11:55:48.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:55:48.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-17T11:55:48.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:55:48.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:55:48.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:55:48.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:55:48.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:55:48.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:55:48.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:55:48.376 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:55:48.376 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:55:48.376 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:55:48.376 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:55:48.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:55:48.523 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:55:48.523 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:55:48.524 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:55:48.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:55:48.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:55:48.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:55:48.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: seq 1 7 2026-02-17T11:55:48.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T11:55:48.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ1 2026-02-17T11:55:48.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ1 2026-02-17T11:55:48.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:55:48.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:55:48.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ1 2026-02-17T11:55:48.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:55:48.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:55:48.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:55:48.895 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:55:48.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:55:49.100 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:55:49.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:55:49.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:55:49.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:55:49.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 1 % 2 2026-02-17T11:55:49.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=1 2026-02-17T11:55:49.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T11:55:49.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3621: corrupt_scrub_erasure: local payload=UVWXYZZZ 2026-02-17T11:55:49.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3622: corrupt_scrub_erasure: echo UVWXYZZZ 2026-02-17T11:55:49.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3623: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:55:49.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:55:49.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:55:49.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:55:49.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:55:49.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:55:49.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:55:49.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:55:49.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:55:49.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:55:49.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:55:49.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:55:49.141 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:55:49.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:55:49.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:55:49.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:55:50.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:55:50.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:55:50.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:55:50.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:55:50.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:55:50.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:55:50.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:55:50.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:55:51.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:55:51.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:55:51.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:55:51.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:55:51.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:55:51.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:55:51.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:55:51.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:55:51.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:55:51.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:55:51.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:55:51.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:55:51.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:55:51.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:55:51.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:55:51.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:55:51.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:55:51.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:55:51.630 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:55:51.630 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:55:51.630 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:55:51.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:55:51.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:55:51.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:55:51.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:55:51.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:55:51.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:55:51.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:55:51.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:55:51.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:55:51.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:55:51.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:55:51.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:55:51.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:55:51.634 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:55:51.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:55:51.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:55:51.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:55:51.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:55:51.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:55:51.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:55:51.653 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:51.644+0000 7f84f8abe8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:51.654 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:51.644+0000 7f84f8abe8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:51.655 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:51.644+0000 7f84f8abe8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:55:51.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:55:51.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:55:51.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:55:51.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:55:51.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:55:51.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:51.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:55:51.790 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:55:51.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:51.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:55:51.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:52.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:52.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:52.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:55:52.942 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:55:52.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:52.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:55:52.970 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:52.960+0000 7f84f8abe8c0 -1 Falling back to public interface 2026-02-17T11:55:53.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:54.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:54.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:54.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:55:54.092 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:55:54.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:54.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:55:54.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:55.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:55.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:55.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:55:55.242 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:55:55.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:55.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:55:55.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:55:55.941 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:55:55.932+0000 7f84f8abe8c0 -1 osd.1 33 log_to_monitors true 2026-02-17T11:55:56.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:55:56.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:55:56.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:55:56.393 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:55:56.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:55:56.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:55:56.540 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 37 up_thru 37 down_at 34 last_clean_interval [10,33) [v2:127.0.0.1:6810/3113749256,v1:127.0.0.1:6811/3113749256] [v2:127.0.0.1:6812/3113749256,v1:127.0.0.1:6813/3113749256] exists,up 77e8fc16-8961-4fc8-a2d2-b3df194c9c30 2026-02-17T11:55:56.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:55:56.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:55:56.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:55:56.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:55:56.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:55:56.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:55:56.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:55:56.541 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:55:56.541 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:55:56.541 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:55:56.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:55:56.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:55:56.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:55:56.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:55:56.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:55:56.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:55:56.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:55:56.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:55:56.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:55:56.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:55:56.808 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:55:56.808 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:55:56.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:55:56.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:55:56.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:55:56.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836490 2026-02-17T11:55:56.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836490 2026-02-17T11:55:56.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836490' 2026-02-17T11:55:56.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:55:56.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:55:56.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789954 2026-02-17T11:55:56.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789954 2026-02-17T11:55:56.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836490 1-158913789954' 2026-02-17T11:55:56.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:55:56.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:55:57.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542151 2026-02-17T11:55:57.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542151 2026-02-17T11:55:57.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836490 1-158913789954 2-60129542151' 2026-02-17T11:55:57.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:55:57.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836490 2026-02-17T11:55:57.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:55:57.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:55:57.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836490 2026-02-17T11:55:57.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:55:57.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836490 2026-02-17T11:55:57.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-17T11:55:57.035 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836490 2026-02-17T11:55:57.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:55:57.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836490 2026-02-17T11:55:57.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:55:58.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:55:58.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:55:58.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836490 2026-02-17T11:55:58.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:55:58.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-158913789954 2026-02-17T11:55:58.353 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:55:58.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:55:58.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-158913789954 2026-02-17T11:55:58.355 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:55:58.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789954 2026-02-17T11:55:58.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 158913789954' 2026-02-17T11:55:58.356 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 158913789954 2026-02-17T11:55:58.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:55:58.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789954 -lt 158913789954 2026-02-17T11:55:58.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:55:58.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542151 2026-02-17T11:55:58.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:55:58.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:55:58.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542151 2026-02-17T11:55:58.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:55:58.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542151 2026-02-17T11:55:58.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542151' 2026-02-17T11:55:58.515 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542151 2026-02-17T11:55:58.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:55:58.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542151 -lt 60129542151 2026-02-17T11:55:58.681 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:55:58.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:55:58.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:55:58.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:55:58.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:55:58.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:55:58.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:55:58.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:55:58.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:55:58.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:55:58.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:55:59.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:55:59.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:55:59.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:55:59.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:55:59.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:55:59.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:55:59.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:55:59.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T11:55:59.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ2 2026-02-17T11:55:59.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ2 2026-02-17T11:55:59.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:55:59.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:55:59.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ2 2026-02-17T11:55:59.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:55:59.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:55:59.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:55:59.407 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:55:59.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:55:59.610 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:55:59.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:55:59.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:55:59.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ2 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:55:59.649 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 2 % 2 2026-02-17T11:55:59.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=0 2026-02-17T11:55:59.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T11:55:59.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3628: corrupt_scrub_erasure: dd if=/dev/urandom of=td/osd-scrub-repair/CORRUPT bs=2048 count=1 2026-02-17T11:55:59.652 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records in 2026-02-17T11:55:59.653 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records out 2026-02-17T11:55:59.653 INFO:tasks.workunit.client.0.trial128.stderr:2048 bytes (2.0 kB, 2.0 KiB) copied, 9.5627e-05 s, 21.4 MB/s 2026-02-17T11:55:59.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3629: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:55:59.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:55:59.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:55:59.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:55:59.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:55:59.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:55:59.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:55:59.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:55:59.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:55:59.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:55:59.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:55:59.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:55:59.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:55:59.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:55:59.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:55:59.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:56:00.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:56:00.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:56:00.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:56:00.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:56:00.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:56:00.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:56:00.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:56:00.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:56:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:56:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:56:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:56:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:56:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:56:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:56:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:56:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:56:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:56:02.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:56:02.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:56:02.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:56:02.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:56:02.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:56:02.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:56:02.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:56:02.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:56:02.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:56:02.155 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:56:02.155 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:56:02.155 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:56:02.156 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:56:02.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:56:02.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:56:02.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:56:02.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:56:02.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:56:02.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:56:02.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:56:02.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:56:02.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:56:02.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:56:02.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:56:02.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:56:02.158 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:56:02.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:56:02.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:56:02.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:56:02.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:56:02.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:56:02.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:56:02.181 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:02.172+0000 7f99d455a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:02.182 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:02.172+0000 7f99d455a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:02.183 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:02.172+0000 7f99d455a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:02.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:56:02.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:56:02.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:56:02.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:56:02.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:56:02.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:02.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:56:02.311 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:56:02.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:02.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:56:02.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:03.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:03.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:03.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:56:03.471 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:56:03.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:03.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:56:03.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:03.673 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:03.664+0000 7f99d455a8c0 -1 Falling back to public interface 2026-02-17T11:56:04.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:04.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:04.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:56:04.628 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:56:04.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:04.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:56:04.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:05.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:05.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:05.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:56:05.787 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:56:05.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:05.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:56:05.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:06.705 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:06.696+0000 7f99d455a8c0 -1 osd.0 41 log_to_monitors true 2026-02-17T11:56:06.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:06.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:06.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:56:06.941 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:56:06.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:06.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:56:07.095 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 45 up_thru 45 down_at 42 last_clean_interval [5,41) [v2:127.0.0.1:6802/213579203,v1:127.0.0.1:6803/213579203] [v2:127.0.0.1:6804/213579203,v1:127.0.0.1:6805/213579203] exists,up 9dcfb325-6c1f-4246-a9b2-e39bf506f0d4 2026-02-17T11:56:07.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:56:07.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:56:07.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:56:07.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:56:07.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:56:07.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:56:07.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:56:07.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:56:07.097 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:56:07.097 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:56:07.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:56:07.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:56:07.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:56:07.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:56:07.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:56:07.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:56:07.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:56:07.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:56:07.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:56:07.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:56:07.365 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:56:07.365 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:56:07.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:56:07.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:56:07.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:56:07.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=193273528322 2026-02-17T11:56:07.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 193273528322 2026-02-17T11:56:07.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-193273528322' 2026-02-17T11:56:07.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:56:07.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:56:07.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789957 2026-02-17T11:56:07.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789957 2026-02-17T11:56:07.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-193273528322 1-158913789957' 2026-02-17T11:56:07.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:56:07.509 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:56:07.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542154 2026-02-17T11:56:07.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542154 2026-02-17T11:56:07.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-193273528322 1-158913789957 2-60129542154' 2026-02-17T11:56:07.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:56:07.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-193273528322 2026-02-17T11:56:07.582 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:56:07.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:56:07.584 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-193273528322 2026-02-17T11:56:07.584 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:56:07.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=193273528322 2026-02-17T11:56:07.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 193273528322' 2026-02-17T11:56:07.585 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 193273528322 2026-02-17T11:56:07.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:56:07.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 193273528322 2026-02-17T11:56:07.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:56:08.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:56:08.749 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:56:08.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528322 -lt 193273528322 2026-02-17T11:56:08.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:56:08.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-158913789957 2026-02-17T11:56:08.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:56:08.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:56:08.907 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-158913789957 2026-02-17T11:56:08.908 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:56:08.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789957 2026-02-17T11:56:08.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 158913789957' 2026-02-17T11:56:08.909 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 158913789957 2026-02-17T11:56:08.910 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:56:09.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789957 -lt 158913789957 2026-02-17T11:56:09.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:56:09.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542154 2026-02-17T11:56:09.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:56:09.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:56:09.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542154 2026-02-17T11:56:09.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:56:09.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542154 2026-02-17T11:56:09.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542154' 2026-02-17T11:56:09.066 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542154 2026-02-17T11:56:09.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:56:09.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542154 -lt 60129542154 2026-02-17T11:56:09.229 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:56:09.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:56:09.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:56:09.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:56:09.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:56:09.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:56:09.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:56:09.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:56:09.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:56:09.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:56:09.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:56:09.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:56:09.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:56:09.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:56:09.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:56:09.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:56:09.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:56:09.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:56:09.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T11:56:09.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ3 2026-02-17T11:56:09.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ3 2026-02-17T11:56:09.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:56:09.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:56:09.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ3 2026-02-17T11:56:09.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:56:09.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:56:09.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:56:09.955 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:56:09.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:56:10.160 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:56:10.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:56:10.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:56:10.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ3 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:56:10.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 3 % 2 2026-02-17T11:56:10.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=1 2026-02-17T11:56:10.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T11:56:10.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3634: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ3 remove 2026-02-17T11:56:10.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:56:10.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:56:10.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:56:10.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:56:10.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ3 remove 2026-02-17T11:56:10.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:56:10.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:56:10.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:56:10.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:56:10.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:56:10.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:56:10.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:56:10.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:56:10.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:56:10.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:56:11.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:56:11.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ3 remove 2026-02-17T11:56:11.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:56:11.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:56:11.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:56:11.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:56:11.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:56:11.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ3 remove 2026-02-17T11:56:12.145 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#3:b197b25d:::EOBJ3:head# 2026-02-17T11:56:12.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:56:12.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:56:12.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:56:12.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:56:12.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:56:12.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:56:12.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:56:12.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:56:12.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:56:12.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:56:12.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:56:12.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:56:12.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:56:12.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:56:12.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:56:12.687 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:56:12.687 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:56:12.687 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:56:12.687 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:56:12.687 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:56:12.687 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:56:12.687 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:56:12.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:56:12.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:56:12.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:56:12.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:56:12.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:56:12.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:56:12.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:56:12.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:56:12.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:56:12.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:56:12.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:56:12.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:56:12.689 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:56:12.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:56:12.690 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:56:12.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:56:12.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:56:12.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:56:12.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:56:12.709 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:12.699+0000 7f79451488c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:12.710 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:12.699+0000 7f79451488c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:12.711 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:12.699+0000 7f79451488c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:12.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:56:12.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:56:12.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:56:12.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:56:12.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:56:12.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:12.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:56:12.848 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:56:12.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:12.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:56:12.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:13.954 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:13.943+0000 7f79451488c0 -1 Falling back to public interface 2026-02-17T11:56:13.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:13.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:13.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:56:13.998 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:56:13.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:13.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:56:14.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:15.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:15.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:15.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:56:15.149 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:56:15.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:15.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:56:15.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:16.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:16.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:16.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:56:16.304 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:56:16.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:16.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:56:16.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:16.950 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:16.939+0000 7f79451488c0 -1 osd.1 49 log_to_monitors true 2026-02-17T11:56:17.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:17.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:17.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:56:17.454 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:56:17.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:17.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:56:17.601 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 53 up_thru 53 down_at 50 last_clean_interval [37,49) [v2:127.0.0.1:6810/723842766,v1:127.0.0.1:6811/723842766] [v2:127.0.0.1:6812/723842766,v1:127.0.0.1:6813/723842766] exists,up 77e8fc16-8961-4fc8-a2d2-b3df194c9c30 2026-02-17T11:56:17.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:56:17.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:56:17.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:56:17.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:56:17.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:56:17.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:56:17.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:56:17.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:56:17.602 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:56:17.603 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:56:17.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:56:17.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:56:17.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:56:17.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:56:17.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:56:17.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:56:17.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:56:17.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:56:17.720 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:56:17.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:56:17.879 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:56:17.879 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:56:17.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:56:17.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:56:17.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:56:17.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=193273528325 2026-02-17T11:56:17.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 193273528325 2026-02-17T11:56:17.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-193273528325' 2026-02-17T11:56:17.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:56:17.954 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:56:18.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=227633266690 2026-02-17T11:56:18.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 227633266690 2026-02-17T11:56:18.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-193273528325 1-227633266690' 2026-02-17T11:56:18.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:56:18.027 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:56:18.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542157 2026-02-17T11:56:18.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542157 2026-02-17T11:56:18.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-193273528325 1-227633266690 2-60129542157' 2026-02-17T11:56:18.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:56:18.099 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-193273528325 2026-02-17T11:56:18.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:56:18.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:56:18.101 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-193273528325 2026-02-17T11:56:18.101 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:56:18.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=193273528325 2026-02-17T11:56:18.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 193273528325' 2026-02-17T11:56:18.103 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 193273528325 2026-02-17T11:56:18.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:56:18.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528324 -lt 193273528325 2026-02-17T11:56:18.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:56:19.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:56:19.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:56:19.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528324 -lt 193273528325 2026-02-17T11:56:19.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:56:20.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:56:20.421 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:56:20.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528325 -lt 193273528325 2026-02-17T11:56:20.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:56:20.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-227633266690 2026-02-17T11:56:20.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:56:20.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:56:20.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-227633266690 2026-02-17T11:56:20.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:56:20.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=227633266690 2026-02-17T11:56:20.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 227633266690' 2026-02-17T11:56:20.579 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 227633266690 2026-02-17T11:56:20.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:56:20.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 227633266690 -lt 227633266690 2026-02-17T11:56:20.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:56:20.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542157 2026-02-17T11:56:20.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:56:20.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:56:20.746 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542157 2026-02-17T11:56:20.746 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:56:20.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542157 2026-02-17T11:56:20.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542157' 2026-02-17T11:56:20.748 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542157 2026-02-17T11:56:20.749 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:56:20.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542158 -lt 60129542157 2026-02-17T11:56:20.906 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:56:20.906 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:56:20.906 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:56:21.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:56:21.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:56:21.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:56:21.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:56:21.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:56:21.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:56:21.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:56:21.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:56:21.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:56:21.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:56:21.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:56:21.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:56:21.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:56:21.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:56:21.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:56:21.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T11:56:21.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ4 2026-02-17T11:56:21.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ4 2026-02-17T11:56:21.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:56:21.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:56:21.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ4 2026-02-17T11:56:21.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:56:21.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:56:21.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:56:21.627 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:56:21.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:56:21.832 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:56:21.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:56:21.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:56:21.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ4 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:56:21.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 4 % 2 2026-02-17T11:56:21.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=0 2026-02-17T11:56:21.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T11:56:21.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3638: corrupt_scrub_erasure: rados --pool ecpool setxattr EOBJ4 key1-EOBJ4 val1-EOBJ4 2026-02-17T11:56:21.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3639: corrupt_scrub_erasure: rados --pool ecpool setxattr EOBJ4 key2-EOBJ4 val2-EOBJ4 2026-02-17T11:56:21.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3642: corrupt_scrub_erasure: echo -n bad-val 2026-02-17T11:56:21.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3643: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-17T11:56:21.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:56:21.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:56:21.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:56:21.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:56:21.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-17T11:56:21.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:56:21.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:56:21.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:56:21.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:56:21.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:56:21.909 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:56:21.909 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:56:21.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:56:21.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:56:21.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:56:23.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:56:23.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-17T11:56:23.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:56:23.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:56:23.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:56:23.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:56:23.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:56:23.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ4 set-attr _key1-EOBJ4 td/osd-scrub-repair/bad-val 2026-02-17T11:56:24.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:56:24.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:56:24.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:56:24.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:56:24.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:56:24.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:56:24.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:56:24.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:56:24.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:56:24.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:56:24.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:56:24.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:56:24.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:56:24.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:56:24.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:56:24.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:56:24.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:56:24.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:56:24.385 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:56:24.385 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:56:24.386 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:56:24.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:56:24.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:56:24.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:56:24.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:56:24.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:56:24.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:56:24.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:56:24.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:56:24.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:56:24.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:56:24.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:56:24.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:56:24.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:56:24.390 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:56:24.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:56:24.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:56:24.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:56:24.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:56:24.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:56:24.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:56:24.409 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:24.399+0000 7f87d5c628c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:24.410 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:24.399+0000 7f87d5c628c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:24.412 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:24.399+0000 7f87d5c628c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:24.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:56:24.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:56:24.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:56:24.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:56:24.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:56:24.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:24.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:56:24.542 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:56:24.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:24.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:56:24.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:25.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:25.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:25.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:56:25.698 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:56:25.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:25.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:56:25.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:25.877 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:25.867+0000 7f87d5c628c0 -1 Falling back to public interface 2026-02-17T11:56:26.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:26.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:26.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:56:26.847 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:56:26.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:26.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:56:27.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:28.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:28.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:28.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:56:28.003 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:56:28.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:28.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:56:28.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:28.865 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:28.855+0000 7f87d5c628c0 -1 osd.0 56 log_to_monitors true 2026-02-17T11:56:29.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:29.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:29.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:56:29.156 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:56:29.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:29.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:56:29.306 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 60 up_thru 60 down_at 57 last_clean_interval [45,56) [v2:127.0.0.1:6802/2053166800,v1:127.0.0.1:6803/2053166800] [v2:127.0.0.1:6804/2053166800,v1:127.0.0.1:6805/2053166800] exists,up 9dcfb325-6c1f-4246-a9b2-e39bf506f0d4 2026-02-17T11:56:29.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:56:29.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:56:29.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:56:29.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:56:29.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:56:29.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:56:29.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:56:29.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:56:29.307 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:56:29.307 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:56:29.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:56:29.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:56:29.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:56:29.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:56:29.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:56:29.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:56:29.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:56:29.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:56:29.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:56:29.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:56:29.584 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:56:29.584 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:56:29.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:56:29.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:56:29.585 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:56:29.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=257698037762 2026-02-17T11:56:29.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 257698037762 2026-02-17T11:56:29.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-257698037762' 2026-02-17T11:56:29.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:56:29.659 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:56:29.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=227633266693 2026-02-17T11:56:29.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 227633266693 2026-02-17T11:56:29.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-257698037762 1-227633266693' 2026-02-17T11:56:29.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:56:29.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:56:29.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542161 2026-02-17T11:56:29.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542161 2026-02-17T11:56:29.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-257698037762 1-227633266693 2-60129542161' 2026-02-17T11:56:29.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:56:29.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-257698037762 2026-02-17T11:56:29.804 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:56:29.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:56:29.805 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-257698037762 2026-02-17T11:56:29.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:56:29.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=257698037762 2026-02-17T11:56:29.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 257698037762' 2026-02-17T11:56:29.807 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 257698037762 2026-02-17T11:56:29.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:56:29.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 257698037762 -lt 257698037762 2026-02-17T11:56:29.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:56:29.959 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-227633266693 2026-02-17T11:56:29.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:56:29.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:56:29.961 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-227633266693 2026-02-17T11:56:29.961 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:56:29.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=227633266693 2026-02-17T11:56:29.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 227633266693' 2026-02-17T11:56:29.963 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 227633266693 2026-02-17T11:56:29.964 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:56:30.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 227633266693 -lt 227633266693 2026-02-17T11:56:30.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:56:30.120 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542161 2026-02-17T11:56:30.121 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:56:30.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:56:30.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542161 2026-02-17T11:56:30.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:56:30.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542161 2026-02-17T11:56:30.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542161' 2026-02-17T11:56:30.124 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542161 2026-02-17T11:56:30.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:56:30.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542160 -lt 60129542161 2026-02-17T11:56:30.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:56:31.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:56:31.285 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:56:31.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542160 -lt 60129542161 2026-02-17T11:56:31.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:56:32.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:56:32.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:56:32.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542161 -lt 60129542161 2026-02-17T11:56:32.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:56:32.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:56:32.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:56:32.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:56:32.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:56:32.782 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:56:32.782 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:56:32.782 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:56:32.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:56:32.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:56:32.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:56:32.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:56:32.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:56:32.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:56:32.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:56:33.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:56:33.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:56:33.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:56:33.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3644: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-17T11:56:33.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:56:33.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:56:33.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T11:56:33.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:56:33.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-17T11:56:33.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:56:33.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:56:33.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T11:56:33.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:56:33.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T11:56:33.129 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:56:33.129 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:56:33.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:56:33.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:56:33.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:56:34.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:56:34.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-17T11:56:34.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:56:34.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:56:34.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T11:56:34.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:56:34.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:56:34.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 EOBJ4 rm-attr _key2-EOBJ4 2026-02-17T11:56:35.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T11:56:35.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:56:35.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:56:35.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T11:56:35.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:56:35.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:56:35.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:56:35.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:56:35.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:56:35.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:56:35.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:56:35.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:56:35.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:56:35.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:56:35.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:56:35.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:56:35.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:56:35.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:56:35.621 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:56:35.622 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:56:35.622 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:56:35.622 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:56:35.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:56:35.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:56:35.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:56:35.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:56:35.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:56:35.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:56:35.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:56:35.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:56:35.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:56:35.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:56:35.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:56:35.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T11:56:35.625 INFO:tasks.workunit.client.0.trial128.stderr:start osd.2 2026-02-17T11:56:35.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:56:35.625 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T11:56:35.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T11:56:35.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:56:35.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:56:35.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:56:35.644 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:35.631+0000 7f2fdc0ad8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:35.645 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:35.635+0000 7f2fdc0ad8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:35.646 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:35.635+0000 7f2fdc0ad8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:35.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T11:56:35.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:56:35.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:56:35.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:56:35.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:56:35.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:35.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:56:35.772 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:56:35.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:35.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:56:35.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:36.878 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:36.867+0000 7f2fdc0ad8c0 -1 Falling back to public interface 2026-02-17T11:56:36.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:36.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:36.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:56:36.929 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:56:36.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:36.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:56:37.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:38.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:38.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:38.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:56:38.082 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:56:38.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:38.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:56:38.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:39.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:39.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:39.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:56:39.240 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:56:39.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:39.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:56:39.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:39.874 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:39.863+0000 7f2fdc0ad8c0 -1 osd.2 62 log_to_monitors true 2026-02-17T11:56:40.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:40.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:40.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:56:40.394 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:56:40.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:40.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:56:40.550 INFO:tasks.workunit.client.0.trial128.stderr:osd.2 up in weight 1 up_from 66 up_thru 66 down_at 63 last_clean_interval [14,62) [v2:127.0.0.1:6818/1714570607,v1:127.0.0.1:6819/1714570607] [v2:127.0.0.1:6820/1714570607,v1:127.0.0.1:6821/1714570607] exists,up 218a6832-4f40-4272-a819-3caf6dbea61c 2026-02-17T11:56:40.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:56:40.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:56:40.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:56:40.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:56:40.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:56:40.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:56:40.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:56:40.551 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:56:40.552 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:56:40.552 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:56:40.552 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:56:40.552 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:56:40.552 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:56:40.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:56:40.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:56:40.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:56:40.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:56:40.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:56:40.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:56:40.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:56:40.815 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:56:40.815 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:56:40.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:56:40.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:56:40.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:56:40.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=257698037765 2026-02-17T11:56:40.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 257698037765 2026-02-17T11:56:40.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-257698037765' 2026-02-17T11:56:40.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:56:40.888 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:56:40.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=227633266696 2026-02-17T11:56:40.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 227633266696 2026-02-17T11:56:40.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-257698037765 1-227633266696' 2026-02-17T11:56:40.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:56:40.959 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:56:41.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=283467841538 2026-02-17T11:56:41.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 283467841538 2026-02-17T11:56:41.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-257698037765 1-227633266696 2-283467841538' 2026-02-17T11:56:41.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:56:41.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-257698037765 2026-02-17T11:56:41.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:56:41.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:56:41.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-257698037765 2026-02-17T11:56:41.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:56:41.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=257698037765 2026-02-17T11:56:41.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 257698037765' 2026-02-17T11:56:41.038 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 257698037765 2026-02-17T11:56:41.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:56:41.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 257698037764 -lt 257698037765 2026-02-17T11:56:41.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:56:42.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:56:42.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:56:42.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 257698037765 -lt 257698037765 2026-02-17T11:56:42.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:56:42.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-227633266696 2026-02-17T11:56:42.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:56:42.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:56:42.353 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-227633266696 2026-02-17T11:56:42.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:56:42.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=227633266696 2026-02-17T11:56:42.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 227633266696' 2026-02-17T11:56:42.355 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 227633266696 2026-02-17T11:56:42.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:56:42.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 227633266696 -lt 227633266696 2026-02-17T11:56:42.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:56:42.524 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-283467841538 2026-02-17T11:56:42.524 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:56:42.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:56:42.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-283467841538 2026-02-17T11:56:42.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:56:42.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=283467841538 2026-02-17T11:56:42.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 283467841538' 2026-02-17T11:56:42.528 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 283467841538 2026-02-17T11:56:42.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:56:42.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 283467841538 -lt 283467841538 2026-02-17T11:56:42.691 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:56:42.691 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:56:42.691 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:56:42.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:56:42.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:56:42.871 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:56:42.871 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:56:42.871 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:56:42.871 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:56:42.871 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:56:42.871 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:56:43.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:56:43.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:56:43.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:56:43.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:56:43.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:56:43.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:56:43.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:56:43.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3645: corrupt_scrub_erasure: echo -n val3-EOBJ4 2026-02-17T11:56:43.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3646: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-17T11:56:43.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:56:43.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:56:43.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T11:56:43.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:56:43.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-17T11:56:43.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:56:43.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:56:43.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T11:56:43.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:56:43.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T11:56:43.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:56:43.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:56:43.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:56:43.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:56:43.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:56:44.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:56:44.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-17T11:56:44.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:56:44.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:56:44.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T11:56:44.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:56:44.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:56:44.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 EOBJ4 set-attr _key3-EOBJ4 td/osd-scrub-repair/newval 2026-02-17T11:56:45.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T11:56:45.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:56:45.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:56:45.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T11:56:45.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:56:45.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T11:56:45.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:56:45.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:56:45.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:56:45.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:56:45.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T11:56:45.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T11:56:45.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:56:45.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:56:45.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:56:45.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:56:45.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:56:45.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:56:45.702 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:56:45.702 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:56:45.702 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:56:45.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:56:45.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:56:45.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:56:45.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:56:45.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:56:45.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:56:45.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:56:45.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:56:45.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:56:45.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:56:45.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:56:45.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T11:56:45.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T11:56:45.704 INFO:tasks.workunit.client.0.trial128.stderr:start osd.2 2026-02-17T11:56:45.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:56:45.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T11:56:45.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T11:56:45.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:56:45.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:56:45.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:56:45.716 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:45.703+0000 7f9000e948c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:45.717 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:45.707+0000 7f9000e948c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:45.718 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:45.707+0000 7f9000e948c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:45.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T11:56:45.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:56:45.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T11:56:45.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:56:45.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:56:45.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:45.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:56:45.861 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:56:45.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:45.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:56:46.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:47.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:47.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:47.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:56:47.011 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:56:47.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:47.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:56:47.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:47.201 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:47.191+0000 7f9000e948c0 -1 Falling back to public interface 2026-02-17T11:56:48.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:48.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:48.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:56:48.161 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:56:48.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:48.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:56:48.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:49.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:49.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:49.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:56:49.314 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:56:49.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:49.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:56:49.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:50.172 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:50.159+0000 7f9000e948c0 -1 osd.2 67 log_to_monitors true 2026-02-17T11:56:50.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:50.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:50.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:56:50.467 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:56:50.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:50.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T11:56:50.621 INFO:tasks.workunit.client.0.trial128.stderr:osd.2 up in weight 1 up_from 71 up_thru 71 down_at 68 last_clean_interval [66,67) [v2:127.0.0.1:6818/1843099357,v1:127.0.0.1:6819/1843099357] [v2:127.0.0.1:6820/1843099357,v1:127.0.0.1:6821/1843099357] exists,up 218a6832-4f40-4272-a819-3caf6dbea61c 2026-02-17T11:56:50.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:56:50.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:56:50.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:56:50.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:56:50.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:56:50.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:56:50.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:56:50.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:56:50.623 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:56:50.623 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:56:50.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:56:50.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:56:50.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:56:50.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:56:50.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:56:50.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:56:50.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:56:50.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:56:50.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:56:50.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:56:50.888 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:56:50.888 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:56:50.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:56:50.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:56:50.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:56:50.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=257698037768 2026-02-17T11:56:50.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 257698037768 2026-02-17T11:56:50.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-257698037768' 2026-02-17T11:56:50.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:56:50.959 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:56:51.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=227633266699 2026-02-17T11:56:51.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 227633266699 2026-02-17T11:56:51.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-257698037768 1-227633266699' 2026-02-17T11:56:51.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:56:51.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:56:51.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678018 2026-02-17T11:56:51.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678018 2026-02-17T11:56:51.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-257698037768 1-227633266699 2-304942678018' 2026-02-17T11:56:51.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:56:51.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-257698037768 2026-02-17T11:56:51.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:56:51.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:56:51.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-257698037768 2026-02-17T11:56:51.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:56:51.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=257698037768 2026-02-17T11:56:51.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 257698037768' 2026-02-17T11:56:51.106 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 257698037768 2026-02-17T11:56:51.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:56:51.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 257698037767 -lt 257698037768 2026-02-17T11:56:51.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:56:52.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:56:52.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:56:52.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 257698037768 -lt 257698037768 2026-02-17T11:56:52.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:56:52.430 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-227633266699 2026-02-17T11:56:52.430 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:56:52.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:56:52.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-227633266699 2026-02-17T11:56:52.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:56:52.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=227633266699 2026-02-17T11:56:52.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 227633266699' 2026-02-17T11:56:52.434 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 227633266699 2026-02-17T11:56:52.434 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:56:52.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 227633266699 -lt 227633266699 2026-02-17T11:56:52.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:56:52.590 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-304942678018 2026-02-17T11:56:52.591 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:56:52.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:56:52.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-304942678018 2026-02-17T11:56:52.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:56:52.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678018 2026-02-17T11:56:52.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 304942678018' 2026-02-17T11:56:52.595 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 304942678018 2026-02-17T11:56:52.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:56:52.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678018 -lt 304942678018 2026-02-17T11:56:52.755 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:56:52.756 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:56:52.756 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:56:52.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:56:52.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:56:52.934 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:56:52.934 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:56:52.934 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:56:52.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:56:52.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:56:52.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:56:53.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:56:53.085 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:56:53.085 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:56:53.085 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:56:53.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:56:53.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:56:53.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:56:53.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3647: corrupt_scrub_erasure: rm td/osd-scrub-repair/bad-val td/osd-scrub-repair/newval 2026-02-17T11:56:53.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T11:56:53.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ5 2026-02-17T11:56:53.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ5 2026-02-17T11:56:53.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:56:53.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:56:53.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ5 2026-02-17T11:56:53.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:56:53.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:56:53.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:56:53.470 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:56:53.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:56:53.675 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:56:53.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:56:53.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:56:53.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ5 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:56:53.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 5 % 2 2026-02-17T11:56:53.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=1 2026-02-17T11:56:53.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T11:56:53.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3652: corrupt_scrub_erasure: dd if=/dev/urandom of=td/osd-scrub-repair/CORRUPT bs=2048 count=2 2026-02-17T11:56:53.709 INFO:tasks.workunit.client.0.trial128.stderr:2+0 records in 2026-02-17T11:56:53.709 INFO:tasks.workunit.client.0.trial128.stderr:2+0 records out 2026-02-17T11:56:53.709 INFO:tasks.workunit.client.0.trial128.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 9.3042e-05 s, 44.0 MB/s 2026-02-17T11:56:53.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3653: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:56:53.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:56:53.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:56:53.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:56:53.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:56:53.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:56:53.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:56:53.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:56:53.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:56:53.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:56:53.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:56:53.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:56:53.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:56:53.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:56:53.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:56:53.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:56:55.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:56:55.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:56:55.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:56:55.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:56:55.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:56:55.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:56:55.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:56:55.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ5 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:56:56.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:56:56.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:56:56.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:56:56.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:56:56.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:56:56.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:56:56.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:56:56.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:56:56.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:56:56.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:56:56.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:56:56.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:56:56.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:56:56.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:56:56.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:56:56.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:56:56.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:56:56.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:56:56.210 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:56:56.210 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:56:56.210 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:56:56.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:56:56.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:56:56.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:56:56.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:56:56.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:56:56.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:56:56.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:56:56.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:56:56.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:56:56.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:56:56.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:56:56.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:56:56.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:56:56.213 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:56:56.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:56:56.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:56:56.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:56:56.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:56:56.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:56:56.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:56:56.233 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:56.222+0000 7fc7eb7188c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:56.234 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:56.222+0000 7fc7eb7188c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:56.235 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:56.222+0000 7fc7eb7188c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:56:56.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:56:56.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:56:56.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:56:56.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:56:56.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:56:56.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:56.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:56:56.365 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:56:56.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:56.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:56:56.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:57.474 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:56:57.462+0000 7fc7eb7188c0 -1 Falling back to public interface 2026-02-17T11:56:57.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:57.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:57.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:56:57.520 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:56:57.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:57.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:56:57.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:58.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:58.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:58.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:56:58.674 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:56:58.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:58.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:56:58.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:56:59.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:56:59.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:56:59.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:56:59.831 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:56:59.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:56:59.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:56:59.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:00.474 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:00.462+0000 7fc7eb7188c0 -1 osd.1 74 log_to_monitors true 2026-02-17T11:57:00.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:00.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:00.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:57:00.988 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:57:00.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:00.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:57:01.143 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 78 up_thru 78 down_at 75 last_clean_interval [53,74) [v2:127.0.0.1:6810/3473585252,v1:127.0.0.1:6811/3473585252] [v2:127.0.0.1:6812/3473585252,v1:127.0.0.1:6813/3473585252] exists,up 77e8fc16-8961-4fc8-a2d2-b3df194c9c30 2026-02-17T11:57:01.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:57:01.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:57:01.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:57:01.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:57:01.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:57:01.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:57:01.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:57:01.144 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:57:01.145 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:57:01.145 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:57:01.145 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:57:01.146 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:57:01.146 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:57:01.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:57:01.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:57:01.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:57:01.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:57:01.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:57:01.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:57:01.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:57:01.415 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:57:01.415 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:57:01.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:57:01.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:01.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:57:01.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=257698037771 2026-02-17T11:57:01.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 257698037771 2026-02-17T11:57:01.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-257698037771' 2026-02-17T11:57:01.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:01.489 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:57:01.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=335007449090 2026-02-17T11:57:01.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 335007449090 2026-02-17T11:57:01.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-257698037771 1-335007449090' 2026-02-17T11:57:01.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:01.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:57:01.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678021 2026-02-17T11:57:01.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678021 2026-02-17T11:57:01.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-257698037771 1-335007449090 2-304942678021' 2026-02-17T11:57:01.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:01.632 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-257698037771 2026-02-17T11:57:01.632 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:01.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:57:01.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-257698037771 2026-02-17T11:57:01.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:01.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=257698037771 2026-02-17T11:57:01.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 257698037771' 2026-02-17T11:57:01.636 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 257698037771 2026-02-17T11:57:01.636 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:57:01.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 257698037770 -lt 257698037771 2026-02-17T11:57:01.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:57:02.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:57:02.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:57:02.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 257698037771 -lt 257698037771 2026-02-17T11:57:02.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:02.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-335007449090 2026-02-17T11:57:02.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:02.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:57:02.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-335007449090 2026-02-17T11:57:02.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:02.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=335007449090 2026-02-17T11:57:02.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 335007449090' 2026-02-17T11:57:02.952 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 335007449090 2026-02-17T11:57:02.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:57:03.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 335007449090 -lt 335007449090 2026-02-17T11:57:03.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:03.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-304942678021 2026-02-17T11:57:03.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:03.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:57:03.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-304942678021 2026-02-17T11:57:03.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:03.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678021 2026-02-17T11:57:03.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 304942678021' 2026-02-17T11:57:03.119 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 304942678021 2026-02-17T11:57:03.119 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:57:03.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678021 -lt 304942678021 2026-02-17T11:57:03.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:57:03.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:57:03.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:57:03.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:57:03.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:57:03.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:57:03.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:57:03.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:57:03.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:57:03.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:57:03.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:57:03.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:57:03.625 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:57:03.625 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:57:03.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:57:03.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:57:03.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:57:03.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:57:03.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T11:57:03.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ6 2026-02-17T11:57:03.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ6 2026-02-17T11:57:03.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:57:03.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:57:03.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ6 2026-02-17T11:57:03.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:57:03.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:57:03.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:57:04.011 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:57:04.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:57:04.215 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:57:04.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:57:04.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:57:04.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ6 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:57:04.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 6 % 2 2026-02-17T11:57:04.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=0 2026-02-17T11:57:04.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T11:57:04.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3657: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-17T11:57:04.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:57:04.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:57:04.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:57:04.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:57:04.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-17T11:57:04.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:57:04.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:57:04.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:57:04.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:57:04.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:57:04.251 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:57:04.251 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:57:04.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:57:04.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:57:04.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:57:05.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:57:05.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ6 rm-attr hinfo_key 2026-02-17T11:57:05.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:57:05.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:57:05.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:57:05.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:57:05.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:57:05.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ6 rm-attr hinfo_key 2026-02-17T11:57:06.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:57:06.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:57:06.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:57:06.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:57:06.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:57:06.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:57:06.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:57:06.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:57:06.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:57:06.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:57:06.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:57:06.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:57:06.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:57:06.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:57:06.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:57:06.749 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:57:06.749 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:57:06.750 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:57:06.750 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:57:06.750 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:57:06.750 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:57:06.750 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:57:06.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:57:06.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:57:06.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:57:06.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:57:06.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:57:06.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:57:06.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:57:06.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:57:06.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:57:06.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:57:06.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:57:06.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:57:06.753 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:57:06.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:57:06.753 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:57:06.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:57:06.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:57:06.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:57:06.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:57:06.773 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:06.762+0000 7efe25f8d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:57:06.774 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:06.762+0000 7efe25f8d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:57:06.776 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:06.762+0000 7efe25f8d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:57:06.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:57:06.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:57:06.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:57:06.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:57:06.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:57:06.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:06.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:57:06.905 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:57:06.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:06.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:57:07.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:08.042 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:08.030+0000 7efe25f8d8c0 -1 Falling back to public interface 2026-02-17T11:57:08.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:08.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:08.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:57:08.062 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:57:08.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:08.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:57:08.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:09.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:09.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:09.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:57:09.215 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:57:09.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:09.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:57:09.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:10.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:10.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:10.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:57:10.372 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:57:10.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:10.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:57:10.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:11.055 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:11.042+0000 7efe25f8d8c0 -1 osd.0 81 log_to_monitors true 2026-02-17T11:57:11.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:11.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:11.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:57:11.531 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:57:11.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:11.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:57:11.684 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 85 up_thru 85 down_at 82 last_clean_interval [60,81) [v2:127.0.0.1:6802/226547839,v1:127.0.0.1:6803/226547839] [v2:127.0.0.1:6804/226547839,v1:127.0.0.1:6805/226547839] exists,up 9dcfb325-6c1f-4246-a9b2-e39bf506f0d4 2026-02-17T11:57:11.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:57:11.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:57:11.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:57:11.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:57:11.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:57:11.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:57:11.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:57:11.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:57:11.686 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:57:11.686 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:57:11.686 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:57:11.686 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:57:11.686 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:57:11.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:57:11.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:57:11.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:57:11.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:57:11.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:57:11.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:57:11.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:57:11.964 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:57:11.964 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:57:11.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:57:11.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:11.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:57:12.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=365072220162 2026-02-17T11:57:12.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 365072220162 2026-02-17T11:57:12.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-365072220162' 2026-02-17T11:57:12.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:12.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:57:12.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=335007449093 2026-02-17T11:57:12.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 335007449093 2026-02-17T11:57:12.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-365072220162 1-335007449093' 2026-02-17T11:57:12.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:12.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:57:12.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678024 2026-02-17T11:57:12.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678024 2026-02-17T11:57:12.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-365072220162 1-335007449093 2-304942678024' 2026-02-17T11:57:12.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:12.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-365072220162 2026-02-17T11:57:12.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:12.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:57:12.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-365072220162 2026-02-17T11:57:12.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:12.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=365072220162 2026-02-17T11:57:12.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 365072220162' 2026-02-17T11:57:12.182 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 365072220162 2026-02-17T11:57:12.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:57:12.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 365072220162 2026-02-17T11:57:12.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:57:13.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:57:13.347 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:57:13.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 365072220162 2026-02-17T11:57:13.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:57:14.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:57:14.500 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:57:14.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 365072220162 -lt 365072220162 2026-02-17T11:57:14.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:14.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-335007449093 2026-02-17T11:57:14.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:14.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:57:14.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-335007449093 2026-02-17T11:57:14.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:14.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=335007449093 2026-02-17T11:57:14.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 335007449093' 2026-02-17T11:57:14.663 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 335007449093 2026-02-17T11:57:14.664 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:57:14.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 335007449093 -lt 335007449093 2026-02-17T11:57:14.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:14.828 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-304942678024 2026-02-17T11:57:14.828 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:14.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:57:14.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-304942678024 2026-02-17T11:57:14.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:14.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678024 2026-02-17T11:57:14.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 304942678024' 2026-02-17T11:57:14.832 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 304942678024 2026-02-17T11:57:14.832 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:57:14.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678024 -lt 304942678024 2026-02-17T11:57:15.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:57:15.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:57:15.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:57:15.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:57:15.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:57:15.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:57:15.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:57:15.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:57:15.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:57:15.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:57:15.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:57:15.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:57:15.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:57:15.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:57:15.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:57:15.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:57:15.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:57:15.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:57:15.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3658: corrupt_scrub_erasure: echo -n bad-val 2026-02-17T11:57:15.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3659: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-17T11:57:15.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:57:15.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:57:15.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:57:15.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:57:15.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-17T11:57:15.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:57:15.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:57:15.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:57:15.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:57:15.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:57:15.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:57:15.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:57:15.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:57:15.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:57:15.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:57:16.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:57:16.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-17T11:57:16.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:57:16.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:57:16.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:57:16.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:57:16.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:57:16.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 EOBJ6 set-attr hinfo_key td/osd-scrub-repair/bad-val 2026-02-17T11:57:18.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:57:18.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:57:18.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:57:18.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:57:18.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:57:18.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:57:18.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:57:18.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:57:18.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:57:18.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:57:18.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:57:18.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:57:18.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:57:18.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:57:18.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:57:18.025 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:57:18.025 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:57:18.025 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:57:18.027 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:57:18.027 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:57:18.027 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:57:18.027 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:57:18.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:57:18.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:57:18.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:57:18.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:57:18.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:57:18.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:57:18.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:57:18.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:57:18.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:57:18.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:57:18.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:57:18.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:57:18.029 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:57:18.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:57:18.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:57:18.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:57:18.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:57:18.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:57:18.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:57:18.049 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:18.038+0000 7f2f1b1218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:57:18.051 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:18.038+0000 7f2f1b1218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:57:18.051 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:18.038+0000 7f2f1b1218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:57:18.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:57:18.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:57:18.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:57:18.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:57:18.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:57:18.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:18.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:57:18.188 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:57:18.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:18.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:57:18.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:19.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:19.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:19.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:57:19.343 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:57:19.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:19.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:57:19.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:19.530 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:19.518+0000 7f2f1b1218c0 -1 Falling back to public interface 2026-02-17T11:57:20.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:20.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:20.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:57:20.505 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:57:20.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:20.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:57:20.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:21.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:21.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:21.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:57:21.665 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:57:21.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:21.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:57:21.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:22.532 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:22.518+0000 7f2f1b1218c0 -1 osd.1 86 log_to_monitors true 2026-02-17T11:57:22.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:22.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:22.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:57:22.821 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:57:22.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:22.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:57:22.979 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 90 up_thru 90 down_at 87 last_clean_interval [78,86) [v2:127.0.0.1:6810/177236867,v1:127.0.0.1:6811/177236867] [v2:127.0.0.1:6812/177236867,v1:127.0.0.1:6813/177236867] exists,up 77e8fc16-8961-4fc8-a2d2-b3df194c9c30 2026-02-17T11:57:22.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:57:22.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:57:22.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:57:22.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:57:22.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:57:22.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:57:22.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:57:22.980 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:57:22.981 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:57:22.981 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:57:22.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:57:22.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:57:22.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:57:23.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:57:23.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:57:23.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:57:23.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:57:23.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:57:23.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:57:23.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:57:23.252 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:57:23.252 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:57:23.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:57:23.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:23.253 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:57:23.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=365072220165 2026-02-17T11:57:23.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 365072220165 2026-02-17T11:57:23.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-365072220165' 2026-02-17T11:57:23.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:23.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:57:23.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=386547056642 2026-02-17T11:57:23.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 386547056642 2026-02-17T11:57:23.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-365072220165 1-386547056642' 2026-02-17T11:57:23.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:23.396 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:57:23.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678027 2026-02-17T11:57:23.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678027 2026-02-17T11:57:23.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-365072220165 1-386547056642 2-304942678027' 2026-02-17T11:57:23.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:23.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-365072220165 2026-02-17T11:57:23.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:23.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:57:23.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-365072220165 2026-02-17T11:57:23.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:23.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=365072220165 2026-02-17T11:57:23.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 365072220165' 2026-02-17T11:57:23.475 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 365072220165 2026-02-17T11:57:23.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:57:23.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 365072220164 -lt 365072220165 2026-02-17T11:57:23.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:57:24.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:57:24.632 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:57:24.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 365072220165 -lt 365072220165 2026-02-17T11:57:24.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:24.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-386547056642 2026-02-17T11:57:24.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:24.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:57:24.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-386547056642 2026-02-17T11:57:24.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:24.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=386547056642 2026-02-17T11:57:24.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 386547056642' 2026-02-17T11:57:24.789 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 386547056642 2026-02-17T11:57:24.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:57:24.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 386547056642 -lt 386547056642 2026-02-17T11:57:24.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:24.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-304942678027 2026-02-17T11:57:24.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:24.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:57:24.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-304942678027 2026-02-17T11:57:24.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:24.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678027 2026-02-17T11:57:24.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 304942678027' 2026-02-17T11:57:24.951 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 304942678027 2026-02-17T11:57:24.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:57:25.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678027 -lt 304942678027 2026-02-17T11:57:25.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:57:25.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:57:25.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:57:25.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:57:25.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:57:25.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:57:25.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:57:25.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:57:25.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:57:25.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:57:25.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:57:25.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:57:25.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:57:25.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:57:25.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:57:25.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:57:25.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:57:25.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:57:25.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3612: corrupt_scrub_erasure: for i in $(seq 1 $total_objs) 2026-02-17T11:57:25.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3613: corrupt_scrub_erasure: objname=EOBJ7 2026-02-17T11:57:25.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3614: corrupt_scrub_erasure: add_something td/osd-scrub-repair ecpool EOBJ7 2026-02-17T11:57:25.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:57:25.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T11:57:25.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=EOBJ7 2026-02-17T11:57:25.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:57:25.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:57:25.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:57:25.814 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:57:25.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:57:26.018 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:57:26.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:57:26.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:57:26.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put EOBJ7 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:57:26.056 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: expr 7 % 2 2026-02-17T11:57:26.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3616: corrupt_scrub_erasure: local osd=1 2026-02-17T11:57:26.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3618: corrupt_scrub_erasure: case $i in 2026-02-17T11:57:26.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3663: corrupt_scrub_erasure: local payload=MAKETHISDIFFERENTFROMOTHEROBJECTS 2026-02-17T11:57:26.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3664: corrupt_scrub_erasure: echo MAKETHISDIFFERENTFROMOTHEROBJECTS 2026-02-17T11:57:26.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3665: corrupt_scrub_erasure: rados --pool ecpool put EOBJ7 td/osd-scrub-repair/DIFFERENT 2026-02-17T11:57:26.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3668: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-17T11:57:26.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:57:26.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:57:26.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:57:26.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:57:26.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-17T11:57:26.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:57:26.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:57:26.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:57:26.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:57:26.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:57:26.081 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:57:26.081 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:57:26.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:57:26.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:57:26.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:57:27.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:57:27.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ1 get-attr hinfo_key 2026-02-17T11:57:27.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:57:27.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:57:27.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:57:27.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:57:27.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:57:27.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ1 get-attr hinfo_key 2026-02-17T11:57:28.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:57:28.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:57:28.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:57:28.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:57:28.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:57:28.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:57:28.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:57:28.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:57:28.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:57:28.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:57:28.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:57:28.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:57:28.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:57:28.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:57:28.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:57:28.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:57:28.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:57:28.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:57:28.238 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:57:28.238 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:57:28.239 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:57:28.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:57:28.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:57:28.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:57:28.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:57:28.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:57:28.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:57:28.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:57:28.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:57:28.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:57:28.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:57:28.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:57:28.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:57:28.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:57:28.240 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:57:28.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:57:28.241 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:57:28.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:57:28.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:57:28.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:57:28.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:57:28.257 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:28.246+0000 7ffb5bfb18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:57:28.259 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:28.246+0000 7ffb5bfb18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:57:28.260 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:28.246+0000 7ffb5bfb18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:57:28.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:57:28.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:57:28.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:57:28.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:57:28.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:57:28.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:28.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:57:28.398 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:57:28.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:28.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:57:28.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:29.486 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:29.474+0000 7ffb5bfb18c0 -1 Falling back to public interface 2026-02-17T11:57:29.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:29.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:29.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:57:29.552 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:57:29.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:29.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:57:29.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:30.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:30.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:30.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:57:30.704 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:57:30.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:30.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:57:30.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:31.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:31.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:31.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:57:31.857 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:57:31.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:31.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:57:32.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:32.762 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:32.750+0000 7ffb5bfb18c0 -1 osd.0 93 log_to_monitors true 2026-02-17T11:57:33.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:33.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:33.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:57:33.009 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:57:33.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:33.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:57:33.160 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 97 up_thru 97 down_at 94 last_clean_interval [85,93) [v2:127.0.0.1:6802/2554081974,v1:127.0.0.1:6803/2554081974] [v2:127.0.0.1:6804/2554081974,v1:127.0.0.1:6805/2554081974] exists,up 9dcfb325-6c1f-4246-a9b2-e39bf506f0d4 2026-02-17T11:57:33.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:57:33.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:57:33.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:57:33.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:57:33.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:57:33.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:57:33.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:57:33.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:57:33.162 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:57:33.162 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:57:33.163 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:57:33.163 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:57:33.163 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:57:33.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:57:33.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:57:33.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:57:33.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:57:33.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:57:33.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:57:33.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:57:33.432 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:57:33.432 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:57:33.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:57:33.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:33.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:57:33.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=416611827714 2026-02-17T11:57:33.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 416611827714 2026-02-17T11:57:33.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-416611827714' 2026-02-17T11:57:33.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:33.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:57:33.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=386547056645 2026-02-17T11:57:33.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 386547056645 2026-02-17T11:57:33.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-416611827714 1-386547056645' 2026-02-17T11:57:33.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:33.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:57:33.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678030 2026-02-17T11:57:33.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678030 2026-02-17T11:57:33.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-416611827714 1-386547056645 2-304942678030' 2026-02-17T11:57:33.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:33.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-416611827714 2026-02-17T11:57:33.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:33.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:57:33.647 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-416611827714 2026-02-17T11:57:33.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:33.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=416611827714 2026-02-17T11:57:33.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 416611827714' 2026-02-17T11:57:33.649 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 416611827714 2026-02-17T11:57:33.649 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:57:33.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 365072220165 -lt 416611827714 2026-02-17T11:57:33.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:57:34.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:57:34.804 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:57:34.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 416611827714 -lt 416611827714 2026-02-17T11:57:34.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:34.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-386547056645 2026-02-17T11:57:34.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:34.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:57:34.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-386547056645 2026-02-17T11:57:34.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:34.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=386547056645 2026-02-17T11:57:34.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 386547056645' 2026-02-17T11:57:34.959 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 386547056645 2026-02-17T11:57:34.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:57:35.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 386547056645 -lt 386547056645 2026-02-17T11:57:35.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:35.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-304942678030 2026-02-17T11:57:35.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:35.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:57:35.128 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-304942678030 2026-02-17T11:57:35.128 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:35.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678030 2026-02-17T11:57:35.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 304942678030' 2026-02-17T11:57:35.130 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 304942678030 2026-02-17T11:57:35.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:57:35.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678030 -lt 304942678030 2026-02-17T11:57:35.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:57:35.294 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:57:35.294 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:57:35.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:57:35.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:57:35.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:57:35.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:57:35.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:57:35.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:57:35.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:57:35.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:57:35.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:57:35.636 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:57:35.637 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:57:35.637 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:57:35.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:57:35.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:57:35.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:57:35.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3669: corrupt_scrub_erasure: objectstore_tool td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-17T11:57:35.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:57:35.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:57:35.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:57:35.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:57:35.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-17T11:57:35.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:57:35.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:57:35.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:57:35.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:57:35.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:57:35.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:57:35.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:57:35.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:57:35.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:57:35.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:57:37.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:57:37.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-17T11:57:37.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:57:37.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:57:37.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:57:37.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:57:37.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:57:37.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 EOBJ7 set-attr hinfo_key td/osd-scrub-repair/hinfo 2026-02-17T11:57:38.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:57:38.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:57:38.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:57:38.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:57:38.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:57:38.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:57:38.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:57:38.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:57:38.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:57:38.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:57:38.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:57:38.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:57:38.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:57:38.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:57:38.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:57:38.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:57:38.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:57:38.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:57:38.310 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:57:38.310 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:57:38.310 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:57:38.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:57:38.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:57:38.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:57:38.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:57:38.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:57:38.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:57:38.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:57:38.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:57:38.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:57:38.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:57:38.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:57:38.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:57:38.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:57:38.313 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:57:38.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:57:38.314 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:57:38.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:57:38.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:57:38.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:57:38.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:57:38.332 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:38.318+0000 7fc99de3e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:57:38.334 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:38.322+0000 7fc99de3e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:57:38.335 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:38.322+0000 7fc99de3e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:57:38.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:57:38.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:57:38.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:57:38.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:57:38.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:57:38.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:38.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:57:38.470 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:57:38.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:38.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:57:38.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:39.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:39.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:39.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:57:39.623 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:57:39.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:57:39.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:39.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:39.842 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:39.830+0000 7fc99de3e8c0 -1 Falling back to public interface 2026-02-17T11:57:40.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:40.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:40.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:57:40.775 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:57:40.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:40.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:57:40.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:41.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:41.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:41.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:57:41.934 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:57:41.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:41.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:57:42.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:57:42.822 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:57:42.809+0000 7fc99de3e8c0 -1 osd.0 98 log_to_monitors true 2026-02-17T11:57:43.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:57:43.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:57:43.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:57:43.086 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:57:43.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:57:43.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:57:43.237 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 102 up_thru 102 down_at 99 last_clean_interval [97,98) [v2:127.0.0.1:6802/1688658844,v1:127.0.0.1:6803/1688658844] [v2:127.0.0.1:6804/1688658844,v1:127.0.0.1:6805/1688658844] exists,up 9dcfb325-6c1f-4246-a9b2-e39bf506f0d4 2026-02-17T11:57:43.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:57:43.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:57:43.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:57:43.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:57:43.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:57:43.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:57:43.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:57:43.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:57:43.239 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:57:43.239 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:57:43.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:57:43.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:57:43.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:57:43.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:57:43.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:57:43.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:57:43.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:57:43.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:57:43.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:57:43.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:57:43.500 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:57:43.500 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:57:43.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:57:43.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:43.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:57:43.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=438086664194 2026-02-17T11:57:43.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 438086664194 2026-02-17T11:57:43.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-438086664194' 2026-02-17T11:57:43.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:43.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:57:43.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=386547056648 2026-02-17T11:57:43.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 386547056648 2026-02-17T11:57:43.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-438086664194 1-386547056648' 2026-02-17T11:57:43.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:43.646 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:57:43.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678033 2026-02-17T11:57:43.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678033 2026-02-17T11:57:43.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-438086664194 1-386547056648 2-304942678033' 2026-02-17T11:57:43.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:43.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-438086664194 2026-02-17T11:57:43.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:43.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:57:43.720 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-438086664194 2026-02-17T11:57:43.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:43.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=438086664194 2026-02-17T11:57:43.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 438086664194' 2026-02-17T11:57:43.722 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 438086664194 2026-02-17T11:57:43.722 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:57:43.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 438086664194 -lt 438086664194 2026-02-17T11:57:43.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:43.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-386547056648 2026-02-17T11:57:43.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:43.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:57:43.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-386547056648 2026-02-17T11:57:43.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:43.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=386547056648 2026-02-17T11:57:43.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 386547056648' 2026-02-17T11:57:43.889 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 386547056648 2026-02-17T11:57:43.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:57:44.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 386547056648 -lt 386547056648 2026-02-17T11:57:44.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:44.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-304942678033 2026-02-17T11:57:44.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:44.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:57:44.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-304942678033 2026-02-17T11:57:44.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:44.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678033 2026-02-17T11:57:44.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 304942678033' 2026-02-17T11:57:44.057 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 304942678033 2026-02-17T11:57:44.058 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:57:44.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678033 -lt 304942678033 2026-02-17T11:57:44.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:57:44.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:57:44.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:57:44.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:57:44.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:57:44.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:57:44.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:57:44.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:57:44.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:57:44.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:57:44.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:57:44.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:57:44.538 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:57:44.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:57:44.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:57:44.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:57:44.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:57:44.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:57:44.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3670: corrupt_scrub_erasure: rm -f td/osd-scrub-repair/hinfo 2026-02-17T11:57:44.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3676: corrupt_scrub_erasure: get_pg ecpool EOBJ0 2026-02-17T11:57:44.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T11:57:44.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=EOBJ0 2026-02-17T11:57:44.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool EOBJ0 2026-02-17T11:57:44.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:57:44.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3676: corrupt_scrub_erasure: local pg=3.0 2026-02-17T11:57:44.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3678: corrupt_scrub_erasure: pg_scrub 3.0 2026-02-17T11:57:44.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=3.0 2026-02-17T11:57:44.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 3.0 2026-02-17T11:57:44.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=3.0 2026-02-17T11:57:44.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T11:57:44.870 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:57:44.870 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:57:44.871 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:57:44.871 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:57:44.871 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:57:45.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T11:57:45.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T11:57:45.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T11:57:45.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T11:57:45.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:57:45.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:57:45.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:57:45.314 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:57:45.314 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:57:45.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:57:45.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:45.314 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:57:45.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=438086664195 2026-02-17T11:57:45.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 438086664195 2026-02-17T11:57:45.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-438086664195' 2026-02-17T11:57:45.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:45.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:57:45.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=386547056649 2026-02-17T11:57:45.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 386547056649 2026-02-17T11:57:45.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-438086664195 1-386547056649' 2026-02-17T11:57:45.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:45.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:57:45.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678035 2026-02-17T11:57:45.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678035 2026-02-17T11:57:45.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-438086664195 1-386547056649 2-304942678035' 2026-02-17T11:57:45.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:45.531 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-438086664195 2026-02-17T11:57:45.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:45.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:57:45.534 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-438086664195 2026-02-17T11:57:45.534 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:45.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=438086664195 2026-02-17T11:57:45.536 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 438086664195 2026-02-17T11:57:45.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 438086664195' 2026-02-17T11:57:45.536 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:57:45.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 438086664194 -lt 438086664195 2026-02-17T11:57:45.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:57:46.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:57:46.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:57:46.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 438086664195 -lt 438086664195 2026-02-17T11:57:46.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:46.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-386547056649 2026-02-17T11:57:46.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:46.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:57:46.858 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-386547056649 2026-02-17T11:57:46.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:46.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=386547056649 2026-02-17T11:57:46.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 386547056649' 2026-02-17T11:57:46.860 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 386547056649 2026-02-17T11:57:46.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:57:47.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 386547056649 -lt 386547056649 2026-02-17T11:57:47.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:47.021 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-304942678035 2026-02-17T11:57:47.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:47.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:57:47.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-304942678035 2026-02-17T11:57:47.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:47.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678035 2026-02-17T11:57:47.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 304942678035' 2026-02-17T11:57:47.025 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 304942678035 2026-02-17T11:57:47.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:57:47.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678035 -lt 304942678035 2026-02-17T11:57:47.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T11:57:47.184 INFO:tasks.workunit.client.0.trial128.stdout:#---------- 3.0 loop 0 2026-02-17T11:57:47.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-17T11:57:47.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 3.0 2026-02-17T11:57:47.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=3.0 2026-02-17T11:57:47.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T11:57:47.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 3.0 query 2026-02-17T11:57:47.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T11:57:47.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T11:57:47.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T11:57:47.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T11:57:47.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T11:57:47.258 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 3.0 2026-02-17T11:57:47.258 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:57:47.258 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:57:47.258 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:57:47.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T11:57:47.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-17T11:55:43.413931+0000 2026-02-17T11:57:47.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 3.0 2026-02-17T11:57:47.556 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 3.0s0 on osd.1 to scrub 2026-02-17T11:57:47.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 3.0 2026-02-17T11:55:43.413931+0000 2026-02-17T11:57:47.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T11:57:47.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:55:43.413931+0000 2026-02-17T11:57:47.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T11:57:47.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:57:47.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:57:47.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T11:57:47.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:57:47.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:57:47.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:57:47.566 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T11:57:47.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:55:43.413931+0000 '>' 2026-02-17T11:55:43.413931+0000 2026-02-17T11:57:47.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:57:48.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:57:48.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:57:48.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T11:57:48.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:57:48.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:57:48.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T11:57:48.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:57:48.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:55:43.413931+0000 '>' 2026-02-17T11:55:43.413931+0000 2026-02-17T11:57:48.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:57:49.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:57:49.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:57:49.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T11:57:49.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:57:49.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:57:49.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T11:57:49.880 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:57:50.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:55:43.413931+0000 '>' 2026-02-17T11:55:43.413931+0000 2026-02-17T11:57:50.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:57:51.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:57:51.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:57:51.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T11:57:51.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:57:51.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:57:51.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T11:57:51.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:57:51.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:55:43.413931+0000 '>' 2026-02-17T11:55:43.413931+0000 2026-02-17T11:57:51.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:57:52.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:57:52.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:57:52.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T11:57:52.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:57:52.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:57:52.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T11:57:52.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:57:52.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:55:43.413931+0000 '>' 2026-02-17T11:55:43.413931+0000 2026-02-17T11:57:52.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:57:53.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:57:53.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:57:53.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T11:57:53.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:57:53.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:57:53.353 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:57:53.353 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T11:57:53.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:55:43.413931+0000 '>' 2026-02-17T11:55:43.413931+0000 2026-02-17T11:57:53.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:57:54.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:57:54.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:57:54.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T11:57:54.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:57:54.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T11:57:54.509 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:57:54.509 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T11:57:54.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:57:48.502026+0000 '>' 2026-02-17T11:55:43.413931+0000 2026-02-17T11:57:54.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:57:54.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3680: corrupt_scrub_erasure: rados list-inconsistent-pg ecpool 2026-02-17T11:57:54.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3682: corrupt_scrub_erasure: jq '. | length' td/osd-scrub-repair/json 2026-02-17T11:57:54.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3682: corrupt_scrub_erasure: test 1 = 1 2026-02-17T11:57:54.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3684: corrupt_scrub_erasure: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-17T11:57:54.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3684: corrupt_scrub_erasure: test 3.0 = 3.0 2026-02-17T11:57:54.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3686: corrupt_scrub_erasure: rados list-inconsistent-obj 3.0 2026-02-17T11:57:54.723 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3688: corrupt_scrub_erasure: jq .epoch td/osd-scrub-repair/json 2026-02-17T11:57:54.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3688: corrupt_scrub_erasure: epoch=102 2026-02-17T11:57:54.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3690: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-17T11:57:54.737 INFO:tasks.workunit.client.0.trial128.stderr: . as $in 2026-02-17T11:57:54.737 INFO:tasks.workunit.client.0.trial128.stderr: | if type == "object" then 2026-02-17T11:57:54.737 INFO:tasks.workunit.client.0.trial128.stderr: reduce keys[] as $key 2026-02-17T11:57:54.737 INFO:tasks.workunit.client.0.trial128.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T11:57:54.738 INFO:tasks.workunit.client.0.trial128.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T11:57:54.738 INFO:tasks.workunit.client.0.trial128.stderr: else f 2026-02-17T11:57:54.738 INFO:tasks.workunit.client.0.trial128.stderr: end; 2026-02-17T11:57:54.738 INFO:tasks.workunit.client.0.trial128.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T11:57:54.738 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T11:57:54.738 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T11:57:54.738 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T11:57:54.738 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-17T11:57:54.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3690: corrupt_scrub_erasure: jq .inconsistents 2026-02-17T11:57:54.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3690: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T11:57:54.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4310: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-17T11:57:54.756 INFO:tasks.workunit.client.0.trial128.stderr: . as $in 2026-02-17T11:57:54.756 INFO:tasks.workunit.client.0.trial128.stderr: | if type == "object" then 2026-02-17T11:57:54.756 INFO:tasks.workunit.client.0.trial128.stderr: reduce keys[] as $key 2026-02-17T11:57:54.756 INFO:tasks.workunit.client.0.trial128.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T11:57:54.756 INFO:tasks.workunit.client.0.trial128.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T11:57:54.757 INFO:tasks.workunit.client.0.trial128.stderr: else f 2026-02-17T11:57:54.757 INFO:tasks.workunit.client.0.trial128.stderr: end; 2026-02-17T11:57:54.757 INFO:tasks.workunit.client.0.trial128.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T11:57:54.757 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T11:57:54.757 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T11:57:54.757 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T11:57:54.757 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-17T11:57:54.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4310: corrupt_scrub_erasure: jq .inconsistents 2026-02-17T11:57:54.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4310: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T11:57:54.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4311: corrupt_scrub_erasure: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T11:57:54.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T11:57:54.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4312: corrupt_scrub_erasure: test no = yes 2026-02-17T11:57:54.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4317: corrupt_scrub_erasure: test '' = yes 2026-02-17T11:57:54.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4322: corrupt_scrub_erasure: pg_deep_scrub 3.0 2026-02-17T11:57:54.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: pg_deep_scrub: local pgid=3.0 2026-02-17T11:57:54.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: pg_deep_scrub: wait_for_pg_clean 3.0 2026-02-17T11:57:54.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=3.0 2026-02-17T11:57:54.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T11:57:54.785 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:57:54.785 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:57:54.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:57:54.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:57:54.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:57:55.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T11:57:55.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T11:57:55.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T11:57:55.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T11:57:55.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:57:55.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:57:55.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:57:55.231 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:57:55.231 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T11:57:55.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:57:55.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:55.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:57:55.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=438086664198 2026-02-17T11:57:55.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 438086664198 2026-02-17T11:57:55.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-438086664198' 2026-02-17T11:57:55.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:55.303 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:57:55.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=386547056652 2026-02-17T11:57:55.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 386547056652 2026-02-17T11:57:55.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-438086664198 1-386547056652' 2026-02-17T11:57:55.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:57:55.374 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T11:57:55.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678038 2026-02-17T11:57:55.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678038 2026-02-17T11:57:55.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-438086664198 1-386547056652 2-304942678038' 2026-02-17T11:57:55.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:55.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-438086664198 2026-02-17T11:57:55.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:55.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:57:55.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-438086664198 2026-02-17T11:57:55.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:55.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=438086664198 2026-02-17T11:57:55.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 438086664198' 2026-02-17T11:57:55.451 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 438086664198 2026-02-17T11:57:55.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:57:55.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 438086664197 -lt 438086664198 2026-02-17T11:57:55.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:57:56.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:57:56.611 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:57:56.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 438086664198 -lt 438086664198 2026-02-17T11:57:56.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:56.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-386547056652 2026-02-17T11:57:56.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:56.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:57:56.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-386547056652 2026-02-17T11:57:56.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:56.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=386547056652 2026-02-17T11:57:56.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 386547056652' 2026-02-17T11:57:56.774 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 386547056652 2026-02-17T11:57:56.774 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:57:56.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 386547056652 -lt 386547056652 2026-02-17T11:57:56.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:57:56.931 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-304942678038 2026-02-17T11:57:56.932 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:57:56.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T11:57:56.934 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-304942678038 2026-02-17T11:57:56.934 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:57:56.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678038 2026-02-17T11:57:56.935 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 304942678038 2026-02-17T11:57:56.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 304942678038' 2026-02-17T11:57:56.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T11:57:57.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678038 -lt 304942678038 2026-02-17T11:57:57.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T11:57:57.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-17T11:57:57.097 INFO:tasks.workunit.client.0.trial128.stdout:#---------- 3.0 loop 0 2026-02-17T11:57:57.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 3.0 2026-02-17T11:57:57.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=3.0 2026-02-17T11:57:57.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T11:57:57.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 3.0 query 2026-02-17T11:57:57.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T11:57:57.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean+inconsistent 2026-02-17T11:57:57.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean+inconsistent == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T11:57:57.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T11:57:57.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T11:57:57.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:57:57.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:57:57.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:57:57.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:57:57.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:57:57.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: local last_scrub=2026-02-17T11:55:43.413931+0000 2026-02-17T11:57:57.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: pg_deep_scrub: ceph pg deep-scrub 3.0 2026-02-17T11:57:57.464 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 3.0s0 on osd.1 to deep-scrub 2026-02-17T11:57:57.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: pg_deep_scrub: wait_for_scrub 3.0 2026-02-17T11:55:43.413931+0000 last_deep_scrub_stamp 2026-02-17T11:57:57.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T11:57:57.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:55:43.413931+0000 2026-02-17T11:57:57.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-17T11:57:57.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:57:57.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:57:57.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:57:57.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:57:57.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:57:57.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:57:57.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:57:57.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:55:43.413931+0000 '>' 2026-02-17T11:55:43.413931+0000 2026-02-17T11:57:57.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:57:58.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:57:58.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:57:58.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:57:58.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:57:58.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:57:58.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:57:58.632 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:57:58.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:55:43.413931+0000 '>' 2026-02-17T11:55:43.413931+0000 2026-02-17T11:57:58.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:57:59.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:57:59.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:57:59.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:57:59.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:57:59.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:57:59.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:57:59.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:57:59.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:55:43.413931+0000 '>' 2026-02-17T11:55:43.413931+0000 2026-02-17T11:57:59.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:58:00.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:58:00.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:58:00.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:58:00.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:58:00.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:58:00.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:58:00.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:58:01.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:55:43.413931+0000 '>' 2026-02-17T11:55:43.413931+0000 2026-02-17T11:58:01.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:58:02.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:58:02.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:58:02.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:58:02.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:58:02.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:58:02.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:58:02.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:58:02.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:55:43.413931+0000 '>' 2026-02-17T11:55:43.413931+0000 2026-02-17T11:58:02.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:58:03.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:58:03.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:58:03.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:58:03.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:58:03.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:58:03.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:58:03.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:58:03.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:55:43.413931+0000 '>' 2026-02-17T11:55:43.413931+0000 2026-02-17T11:58:03.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:58:04.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:58:04.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:58:04.409 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:58:04.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:58:04.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:58:04.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:58:04.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:58:04.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:57:58.399677+0000 '>' 2026-02-17T11:55:43.413931+0000 2026-02-17T11:58:04.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:58:04.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4324: corrupt_scrub_erasure: rados list-inconsistent-pg ecpool 2026-02-17T11:58:04.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4326: corrupt_scrub_erasure: jq '. | length' td/osd-scrub-repair/json 2026-02-17T11:58:04.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4326: corrupt_scrub_erasure: test 1 = 1 2026-02-17T11:58:04.590 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4328: corrupt_scrub_erasure: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-17T11:58:04.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4328: corrupt_scrub_erasure: test 3.0 = 3.0 2026-02-17T11:58:04.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4330: corrupt_scrub_erasure: rados list-inconsistent-obj 3.0 2026-02-17T11:58:04.617 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4332: corrupt_scrub_erasure: jq .epoch td/osd-scrub-repair/json 2026-02-17T11:58:04.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4332: corrupt_scrub_erasure: epoch=102 2026-02-17T11:58:04.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4334: corrupt_scrub_erasure: '[' true = true ']' 2026-02-17T11:58:04.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4336: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-17T11:58:04.628 INFO:tasks.workunit.client.0.trial128.stderr: . as $in 2026-02-17T11:58:04.628 INFO:tasks.workunit.client.0.trial128.stderr: | if type == "object" then 2026-02-17T11:58:04.628 INFO:tasks.workunit.client.0.trial128.stderr: reduce keys[] as $key 2026-02-17T11:58:04.628 INFO:tasks.workunit.client.0.trial128.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T11:58:04.628 INFO:tasks.workunit.client.0.trial128.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T11:58:04.628 INFO:tasks.workunit.client.0.trial128.stderr: else f 2026-02-17T11:58:04.629 INFO:tasks.workunit.client.0.trial128.stderr: end; 2026-02-17T11:58:04.629 INFO:tasks.workunit.client.0.trial128.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T11:58:04.629 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T11:58:04.629 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T11:58:04.629 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T11:58:04.629 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-17T11:58:04.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4336: corrupt_scrub_erasure: jq .inconsistents 2026-02-17T11:58:04.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:4336: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T11:58:04.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5724: corrupt_scrub_erasure: jq 'def walk(f): 2026-02-17T11:58:04.646 INFO:tasks.workunit.client.0.trial128.stderr: . as $in 2026-02-17T11:58:04.646 INFO:tasks.workunit.client.0.trial128.stderr: | if type == "object" then 2026-02-17T11:58:04.646 INFO:tasks.workunit.client.0.trial128.stderr: reduce keys[] as $key 2026-02-17T11:58:04.646 INFO:tasks.workunit.client.0.trial128.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T11:58:04.646 INFO:tasks.workunit.client.0.trial128.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T11:58:04.646 INFO:tasks.workunit.client.0.trial128.stderr: else f 2026-02-17T11:58:04.646 INFO:tasks.workunit.client.0.trial128.stderr: end; 2026-02-17T11:58:04.646 INFO:tasks.workunit.client.0.trial128.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T11:58:04.646 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T11:58:04.646 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T11:58:04.647 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T11:58:04.647 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-17T11:58:04.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5724: corrupt_scrub_erasure: jq .inconsistents 2026-02-17T11:58:04.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5724: corrupt_scrub_erasure: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T11:58:04.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5725: corrupt_scrub_erasure: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T11:58:04.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T11:58:04.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5726: corrupt_scrub_erasure: test no = yes 2026-02-17T11:58:04.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5737: corrupt_scrub_erasure: test '' = yes 2026-02-17T11:58:04.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5742: corrupt_scrub_erasure: ceph osd pool rm ecpool ecpool --yes-i-really-really-mean-it 2026-02-17T11:58:04.871 INFO:tasks.workunit.client.0.trial128.stderr:pool 'ecpool' removed 2026-02-17T11:58:04.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T11:58:04.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:58:04.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:58:04.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:58:04.884 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:58:04.884 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:58:04.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:58:04.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:58:04.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:58:05.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:58:05.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:58:05.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:58:05.198 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:58:05.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:58:05.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:58:05.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:58:05.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:58:05.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:58:05.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:58:05.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:58:05.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:58:05.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:58:05.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:58:05.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:58:05.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:58:05.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:58:05.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:58:05.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:58:05.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:58:05.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T11:58:05.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T11:58:05.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T11:58:05.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T11:58:05.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T11:58:05.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T11:58:05.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T11:58:05.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:58:05.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:58:05.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:58:05.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:58:05.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:58:05.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:58:05.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T11:58:05.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T11:58:05.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T11:58:05.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T11:58:05.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T11:58:05.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T11:58:05.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:58:05.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T11:58:05.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T11:58:05.242 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T11:58:05.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T11:58:05.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T11:58:05.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T11:58:05.247 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T11:58:05.247 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:58:05.247 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:58:05.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T11:58:05.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T11:58:05.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T11:58:05.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T11:58:05.250 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T11:58:05.250 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:58:05.250 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:58:05.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T11:58:05.251 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T11:58:05.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T11:58:05.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T11:58:05.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T11:58:05.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_scrub_replicated td/osd-scrub-repair 2026-02-17T11:58:05.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:991: TEST_corrupt_scrub_replicated: local dir=td/osd-scrub-repair 2026-02-17T11:58:05.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:992: TEST_corrupt_scrub_replicated: local poolname=csr_pool 2026-02-17T11:58:05.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:993: TEST_corrupt_scrub_replicated: local total_objs=19 2026-02-17T11:58:05.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:995: TEST_corrupt_scrub_replicated: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-17T11:58:05.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T11:58:05.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T11:58:05.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T11:58:05.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T11:58:05.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T11:58:05.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-17T11:58:05.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T11:58:05.276 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:58:05.276 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:58:05.276 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:58:05.276 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:58:05.276 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:58:05.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:58:05.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T11:58:05.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T11:58:05.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T11:58:05.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:58:05.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:58:05.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T11:58:05.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T11:58:05.306 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:58:05.306 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:58:05.306 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:58:05.306 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:58:05.307 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:58:05.307 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:58:05.307 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:58:05.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:58:05.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T11:58:05.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T11:58:05.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T11:58:05.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T11:58:05.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T11:58:05.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T11:58:05.364 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T11:58:05.364 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T11:58:05.364 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T11:58:05.365 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:58:05.365 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:58:05.366 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:58:05.366 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T11:58:05.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T11:58:05.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T11:58:05.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:996: TEST_corrupt_scrub_replicated: run_mgr td/osd-scrub-repair x 2026-02-17T11:58:05.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T11:58:05.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T11:58:05.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T11:58:05.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T11:58:05.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T11:58:05.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T11:58:05.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T11:58:05.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:58:05.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:58:05.517 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:58:05.517 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:58:05.517 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:58:05.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:58:05.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:58:05.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T11:58:05.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:997: TEST_corrupt_scrub_replicated: run_osd td/osd-scrub-repair 0 2026-02-17T11:58:05.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:58:05.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:58:05.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T11:58:05.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:58:05.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:58:05.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:58:05.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:58:05.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:58:05.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:58:05.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:58:05.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:58:05.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:58:05.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:58:05.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:58:05.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:58:05.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:58:05.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:58:05.542 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:58:05.542 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:58:05.542 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:58:05.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:58:05.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:58:05.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:58:05.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:58:05.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:58:05.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:58:05.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:58:05.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:58:05.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:58:05.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:58:05.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:58:05.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:58:05.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:58:05.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:58:05.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:58:05.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=298610f2-1d9b-4f6c-b9da-00d205e834ad 2026-02-17T11:58:05.545 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 298610f2-1d9b-4f6c-b9da-00d205e834ad 2026-02-17T11:58:05.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 298610f2-1d9b-4f6c-b9da-00d205e834ad' 2026-02-17T11:58:05.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:58:05.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDNV5RpnJmWIBAAzWyrwSi1t2dvQ2H1MR+XQg== 2026-02-17T11:58:05.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDNV5RpnJmWIBAAzWyrwSi1t2dvQ2H1MR+XQg=="}' 2026-02-17T11:58:05.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 298610f2-1d9b-4f6c-b9da-00d205e834ad -i td/osd-scrub-repair/0/new.json 2026-02-17T11:58:05.644 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:58:05.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T11:58:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDNV5RpnJmWIBAAzWyrwSi1t2dvQ2H1MR+XQg== --osd-uuid 298610f2-1d9b-4f6c-b9da-00d205e834ad 2026-02-17T11:58:05.669 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:05.661+0000 7f21cd2408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:58:05.670 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:05.661+0000 7f21cd2408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:58:05.671 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:05.661+0000 7f21cd2408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:58:05.672 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:05.661+0000 7f21cd2408c0 -1 bdev(0x56184c2f1c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T11:58:05.672 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:05.661+0000 7f21cd2408c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T11:58:08.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T11:58:08.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:58:08.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T11:58:08.323 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T11:58:08.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:58:08.522 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T11:58:08.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T11:58:08.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:58:08.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:58:08.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:58:08.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:58:08.538 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:08.529+0000 7f6dcd3328c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:58:08.539 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:08.529+0000 7f6dcd3328c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:58:08.540 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:08.533+0000 7f6dcd3328c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:58:08.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T11:58:08.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:58:08.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:58:08.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:58:08.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:58:08.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:08.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:58:08.676 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:58:08.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:08.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:58:08.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:58:09.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:58:09.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:09.834 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:58:09.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:58:09.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:09.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:58:09.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:58:10.254 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:10.245+0000 7f6dcd3328c0 -1 Falling back to public interface 2026-02-17T11:58:10.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:58:10.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:10.998 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:58:10.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:58:10.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:10.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:58:11.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:58:12.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:58:12.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:12.153 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:58:12.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:58:12.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:12.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:58:12.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:58:13.203 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:13.193+0000 7f6dcd3328c0 -1 osd.0 0 log_to_monitors true 2026-02-17T11:58:13.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:58:13.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:13.310 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:58:13.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:58:13.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:13.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:58:13.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:58:14.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:58:14.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:14.471 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:58:14.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:58:14.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:14.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:58:14.625 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/980636368,v1:127.0.0.1:6803/980636368] [v2:127.0.0.1:6804/980636368,v1:127.0.0.1:6805/980636368] exists,up 298610f2-1d9b-4f6c-b9da-00d205e834ad 2026-02-17T11:58:14.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:58:14.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:58:14.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:58:14.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:998: TEST_corrupt_scrub_replicated: run_osd td/osd-scrub-repair 1 2026-02-17T11:58:14.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T11:58:14.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T11:58:14.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T11:58:14.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T11:58:14.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:58:14.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:58:14.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:58:14.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:58:14.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:58:14.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:58:14.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:58:14.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T11:58:14.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T11:58:14.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:58:14.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T11:58:14.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:58:14.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:58:14.631 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:58:14.631 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:58:14.631 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:58:14.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:58:14.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:58:14.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:58:14.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T11:58:14.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T11:58:14.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:58:14.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:58:14.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:58:14.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:58:14.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:58:14.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:58:14.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T11:58:14.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T11:58:14.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:58:14.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T11:58:14.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T11:58:14.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 e810ce24-5a23-4965-9d34-3976c6e1a6f8' 2026-02-17T11:58:14.634 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T11:58:14.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T11:58:14.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDWV5RpNPrgJRAA0yKxQ5HlRto2pcuwJUKKWg== 2026-02-17T11:58:14.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDWV5RpNPrgJRAA0yKxQ5HlRto2pcuwJUKKWg=="}' 2026-02-17T11:58:14.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e810ce24-5a23-4965-9d34-3976c6e1a6f8 -i td/osd-scrub-repair/1/new.json 2026-02-17T11:58:14.801 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:58:14.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T11:58:14.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDWV5RpNPrgJRAA0yKxQ5HlRto2pcuwJUKKWg== --osd-uuid e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T11:58:14.826 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:14.817+0000 7f1695b538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:58:14.828 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:14.817+0000 7f1695b538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:58:14.829 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:14.821+0000 7f1695b538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:58:14.829 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:14.821+0000 7f1695b538c0 -1 bdev(0x55f794aafc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T11:58:14.829 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:14.821+0000 7f1695b538c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T11:58:17.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T11:58:17.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T11:58:17.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T11:58:17.099 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T11:58:17.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T11:58:17.298 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T11:58:17.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T11:58:17.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:58:17.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T11:58:17.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T11:58:17.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T11:58:17.321 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:17.313+0000 7fa1866528c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:58:17.323 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:17.313+0000 7fa1866528c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:58:17.324 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:17.317+0000 7fa1866528c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:58:17.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T11:58:17.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:58:17.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:58:17.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:58:17.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:58:17.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:17.457 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T11:58:17.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:58:17.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:17.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:58:17.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:58:18.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:58:18.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:18.613 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T11:58:18.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:58:18.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:18.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:58:18.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:58:18.825 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:18.817+0000 7fa1866528c0 -1 Falling back to public interface 2026-02-17T11:58:19.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:58:19.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:19.770 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T11:58:19.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:58:19.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:19.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:58:19.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:58:20.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:58:20.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:20.929 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T11:58:20.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:58:20.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:20.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:58:21.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:58:21.813 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:21.805+0000 7fa1866528c0 -1 osd.1 0 log_to_monitors true 2026-02-17T11:58:22.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:58:22.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:22.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:58:22.086 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T11:58:22.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:22.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:58:22.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:58:23.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:58:23.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:23.235 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T11:58:23.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T11:58:23.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:23.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:58:23.387 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4083418836,v1:127.0.0.1:6811/4083418836] [v2:127.0.0.1:6812/4083418836,v1:127.0.0.1:6813/4083418836] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T11:58:23.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:58:23.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:58:23.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:58:23.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:999: TEST_corrupt_scrub_replicated: create_rbd_pool 2026-02-17T11:58:23.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T11:58:23.531 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' does not exist 2026-02-17T11:58:23.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T11:58:23.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T11:58:23.741 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T11:58:23.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:58:24.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T11:58:25.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1000: TEST_corrupt_scrub_replicated: wait_for_clean 2026-02-17T11:58:25.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:58:25.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:58:25.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:58:25.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:58:25.036 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:58:25.036 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:58:25.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:58:25.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:58:25.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:58:25.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:58:25.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:58:25.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:58:25.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:58:25.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:58:25.149 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:58:25.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:58:25.303 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:58:25.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:58:25.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:58:25.303 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:58:25.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T11:58:25.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T11:58:25.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T11:58:25.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:58:25.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:58:25.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672962 2026-02-17T11:58:25.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672962 2026-02-17T11:58:25.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-42949672962' 2026-02-17T11:58:25.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:58:25.445 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T11:58:25.445 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:58:25.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:58:25.448 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T11:58:25.448 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:58:25.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T11:58:25.450 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836484 2026-02-17T11:58:25.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T11:58:25.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:58:25.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T11:58:25.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:58:26.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:58:26.614 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:58:26.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T11:58:26.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:58:26.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672962 2026-02-17T11:58:26.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:58:26.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:58:26.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672962 2026-02-17T11:58:26.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:58:26.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672962 2026-02-17T11:58:26.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672962' 2026-02-17T11:58:26.772 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672962 2026-02-17T11:58:26.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:58:26.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672962 -lt 42949672962 2026-02-17T11:58:26.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:58:26.928 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:58:26.928 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:58:27.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T11:58:27.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:58:27.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:58:27.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:58:27.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:58:27.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:58:27.110 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:58:27.110 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:58:27.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T11:58:27.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:58:27.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:58:27.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:58:27.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T11:58:27.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:58:27.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:58:27.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1002: TEST_corrupt_scrub_replicated: create_pool foo 1 2026-02-17T11:58:27.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 1 2026-02-17T11:58:27.682 INFO:tasks.workunit.client.0.trial128.stderr:pool 'foo' created 2026-02-17T11:58:27.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:58:28.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1003: TEST_corrupt_scrub_replicated: create_pool csr_pool 1 1 2026-02-17T11:58:28.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create csr_pool 1 1 2026-02-17T11:58:28.901 INFO:tasks.workunit.client.0.trial128.stderr:pool 'csr_pool' created 2026-02-17T11:58:28.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T11:58:29.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1004: TEST_corrupt_scrub_replicated: wait_for_clean 2026-02-17T11:58:29.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:58:29.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:58:29.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:58:29.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:58:29.917 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:58:29.917 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:58:29.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:58:29.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:58:29.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:58:30.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:58:30.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:58:30.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:58:30.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:58:30.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:58:30.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:58:30.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:58:30.189 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:58:30.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:58:30.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:58:30.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:58:30.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836486 2026-02-17T11:58:30.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836486 2026-02-17T11:58:30.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486' 2026-02-17T11:58:30.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:58:30.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:58:30.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T11:58:30.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T11:58:30.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-42949672964' 2026-02-17T11:58:30.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:58:30.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836486 2026-02-17T11:58:30.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:58:30.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:58:30.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836486 2026-02-17T11:58:30.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:58:30.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836486 2026-02-17T11:58:30.339 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836486 2026-02-17T11:58:30.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-17T11:58:30.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:58:30.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-17T11:58:30.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:58:31.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:58:31.497 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:58:31.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-17T11:58:31.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:58:32.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:58:32.649 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:58:32.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-17T11:58:32.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:58:32.807 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T11:58:32.807 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:58:32.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:58:32.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T11:58:32.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:58:32.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T11:58:32.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T11:58:32.811 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672964 2026-02-17T11:58:32.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:58:32.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T11:58:32.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:58:32.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:58:32.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:58:33.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:58:33.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:58:33.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:58:33.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:58:33.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:58:33.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:58:33.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:58:33.151 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:58:33.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:58:33.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:58:33.299 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:58:33.299 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:58:33.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:58:33.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:58:33.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:58:33.476 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: seq 1 19 2026-02-17T11:58:33.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:33.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ1 2026-02-17T11:58:33.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ1 2026-02-17T11:58:33.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:33.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:33.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ1 2026-02-17T11:58:33.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:33.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:33.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:33.719 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:33.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:33.922 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:33.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:33.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:33.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:33.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ1 hdr-ROBJ1 2026-02-17T11:58:33.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ1 key-ROBJ1 val-ROBJ1 2026-02-17T11:58:34.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:34.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ2 2026-02-17T11:58:34.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ2 2026-02-17T11:58:34.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:34.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:34.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ2 2026-02-17T11:58:34.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:34.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:34.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:34.203 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:34.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:34.408 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:34.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:34.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:34.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ2 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:34.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ2 hdr-ROBJ2 2026-02-17T11:58:34.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ2 key-ROBJ2 val-ROBJ2 2026-02-17T11:58:34.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:34.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ3 2026-02-17T11:58:34.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ3 2026-02-17T11:58:34.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:34.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:34.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ3 2026-02-17T11:58:34.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:34.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:34.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:34.711 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:34.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:34.913 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:34.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:34.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:34.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ3 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:34.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ3 hdr-ROBJ3 2026-02-17T11:58:34.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ3 key-ROBJ3 val-ROBJ3 2026-02-17T11:58:35.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:35.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ4 2026-02-17T11:58:35.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ4 2026-02-17T11:58:35.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:35.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:35.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ4 2026-02-17T11:58:35.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:35.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:35.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:35.208 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:35.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:35.414 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:35.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:35.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:35.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ4 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:35.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ4 hdr-ROBJ4 2026-02-17T11:58:35.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ4 key-ROBJ4 val-ROBJ4 2026-02-17T11:58:35.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:35.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ5 2026-02-17T11:58:35.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ5 2026-02-17T11:58:35.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:35.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:35.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ5 2026-02-17T11:58:35.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:35.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:35.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:35.692 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:35.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:35.899 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:35.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:35.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:35.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ5 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:35.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ5 hdr-ROBJ5 2026-02-17T11:58:35.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ5 key-ROBJ5 val-ROBJ5 2026-02-17T11:58:35.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:35.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ6 2026-02-17T11:58:35.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ6 2026-02-17T11:58:35.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:35.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:35.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ6 2026-02-17T11:58:35.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:35.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:35.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:36.187 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:36.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:36.393 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:36.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:36.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:36.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ6 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:36.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ6 hdr-ROBJ6 2026-02-17T11:58:36.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ6 key-ROBJ6 val-ROBJ6 2026-02-17T11:58:36.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:36.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ7 2026-02-17T11:58:36.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ7 2026-02-17T11:58:36.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:36.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:36.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ7 2026-02-17T11:58:36.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:36.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:36.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:36.696 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:36.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:36.898 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:36.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:36.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:36.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ7 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:36.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ7 hdr-ROBJ7 2026-02-17T11:58:36.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ7 key-ROBJ7 val-ROBJ7 2026-02-17T11:58:36.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:36.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ8 2026-02-17T11:58:36.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ8 2026-02-17T11:58:36.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:36.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:36.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ8 2026-02-17T11:58:36.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:36.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:36.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:37.184 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:37.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:37.389 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:37.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:37.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:37.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ8 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:37.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ8 hdr-ROBJ8 2026-02-17T11:58:37.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ8 key-ROBJ8 val-ROBJ8 2026-02-17T11:58:37.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:37.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ9 2026-02-17T11:58:37.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ9 2026-02-17T11:58:37.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:37.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:37.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ9 2026-02-17T11:58:37.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:37.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:37.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:37.667 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:37.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:37.873 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:37.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:37.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:37.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ9 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:37.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ9 hdr-ROBJ9 2026-02-17T11:58:37.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ9 key-ROBJ9 val-ROBJ9 2026-02-17T11:58:37.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:37.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ10 2026-02-17T11:58:37.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ10 2026-02-17T11:58:37.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:37.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:37.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ10 2026-02-17T11:58:37.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:37.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:37.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:38.145 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:38.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:38.350 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:38.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:38.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:38.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ10 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:38.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ10 hdr-ROBJ10 2026-02-17T11:58:38.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ10 key-ROBJ10 val-ROBJ10 2026-02-17T11:58:38.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:38.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ11 2026-02-17T11:58:38.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ11 2026-02-17T11:58:38.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:38.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:38.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ11 2026-02-17T11:58:38.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:38.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:38.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:38.654 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:38.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:38.857 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:38.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:38.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:38.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ11 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:38.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ11 hdr-ROBJ11 2026-02-17T11:58:38.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ11 key-ROBJ11 val-ROBJ11 2026-02-17T11:58:38.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:38.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ12 2026-02-17T11:58:38.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ12 2026-02-17T11:58:38.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:38.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:38.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ12 2026-02-17T11:58:38.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:38.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:38.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:39.147 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:39.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:39.353 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:39.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:39.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:39.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ12 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:39.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ12 hdr-ROBJ12 2026-02-17T11:58:39.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ12 key-ROBJ12 val-ROBJ12 2026-02-17T11:58:39.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:39.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ13 2026-02-17T11:58:39.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ13 2026-02-17T11:58:39.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:39.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:39.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ13 2026-02-17T11:58:39.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:39.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:39.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:39.645 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:39.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:39.851 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:39.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:39.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:39.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ13 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:39.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ13 hdr-ROBJ13 2026-02-17T11:58:39.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ13 key-ROBJ13 val-ROBJ13 2026-02-17T11:58:39.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:39.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ14 2026-02-17T11:58:39.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ14 2026-02-17T11:58:39.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:39.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:39.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ14 2026-02-17T11:58:39.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:39.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:39.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:40.137 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:40.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:40.342 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:40.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:40.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:40.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ14 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:40.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ14 hdr-ROBJ14 2026-02-17T11:58:40.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ14 key-ROBJ14 val-ROBJ14 2026-02-17T11:58:40.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:40.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ15 2026-02-17T11:58:40.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ15 2026-02-17T11:58:40.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:40.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:40.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ15 2026-02-17T11:58:40.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:40.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:40.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:40.645 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:40.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:40.848 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:40.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:40.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:40.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ15 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:40.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ15 hdr-ROBJ15 2026-02-17T11:58:40.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ15 key-ROBJ15 val-ROBJ15 2026-02-17T11:58:40.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:40.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ16 2026-02-17T11:58:40.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ16 2026-02-17T11:58:40.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:40.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:40.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ16 2026-02-17T11:58:40.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:40.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:40.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:41.138 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:41.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:41.343 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:41.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:41.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:41.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ16 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:41.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ16 hdr-ROBJ16 2026-02-17T11:58:41.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ16 key-ROBJ16 val-ROBJ16 2026-02-17T11:58:41.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:41.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ17 2026-02-17T11:58:41.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ17 2026-02-17T11:58:41.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:41.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:41.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ17 2026-02-17T11:58:41.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:41.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:41.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:41.635 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:41.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:41.840 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:41.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:41.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:41.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ17 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:41.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ17 hdr-ROBJ17 2026-02-17T11:58:41.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ17 key-ROBJ17 val-ROBJ17 2026-02-17T11:58:41.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:41.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ18 2026-02-17T11:58:41.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ18 2026-02-17T11:58:41.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:41.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:41.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ18 2026-02-17T11:58:41.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:41.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:41.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:42.132 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:42.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:42.337 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:42.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:42.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:42.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ18 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:42.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ18 hdr-ROBJ18 2026-02-17T11:58:42.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ18 key-ROBJ18 val-ROBJ18 2026-02-17T11:58:42.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1006: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:42.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1007: TEST_corrupt_scrub_replicated: objname=ROBJ19 2026-02-17T11:58:42.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1008: TEST_corrupt_scrub_replicated: add_something td/osd-scrub-repair csr_pool ROBJ19 2026-02-17T11:58:42.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T11:58:42.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T11:58:42.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ19 2026-02-17T11:58:42.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T11:58:42.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T11:58:42.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T11:58:42.618 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T11:58:42.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T11:58:42.821 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T11:58:42.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T11:58:42.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T11:58:42.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ19 td/osd-scrub-repair/ORIGINAL 2026-02-17T11:58:42.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1010: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapheader ROBJ19 hdr-ROBJ19 2026-02-17T11:58:42.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1011: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ19 key-ROBJ19 val-ROBJ19 2026-02-17T11:58:42.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1015: TEST_corrupt_scrub_replicated: dd if=/dev/zero of=td/osd-scrub-repair/new.ROBJ19 bs=1024 count=1025 2026-02-17T11:58:42.909 INFO:tasks.workunit.client.0.trial128.stderr:1025+0 records in 2026-02-17T11:58:42.909 INFO:tasks.workunit.client.0.trial128.stderr:1025+0 records out 2026-02-17T11:58:42.909 INFO:tasks.workunit.client.0.trial128.stderr:1049600 bytes (1.0 MB, 1.0 MiB) copied, 0.00416465 s, 252 MB/s 2026-02-17T11:58:42.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1016: TEST_corrupt_scrub_replicated: rados --pool csr_pool put ROBJ19 td/osd-scrub-repair/new.ROBJ19 2026-02-17T11:58:42.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1017: TEST_corrupt_scrub_replicated: rm -f td/osd-scrub-repair/new.ROBJ19 2026-02-17T11:58:42.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1019: TEST_corrupt_scrub_replicated: get_pg csr_pool ROBJ0 2026-02-17T11:58:42.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=csr_pool 2026-02-17T11:58:42.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=ROBJ0 2026-02-17T11:58:42.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map csr_pool ROBJ0 2026-02-17T11:58:42.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T11:58:43.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1019: TEST_corrupt_scrub_replicated: local pg=3.0 2026-02-17T11:58:43.087 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1020: TEST_corrupt_scrub_replicated: get_primary csr_pool ROBJ0 2026-02-17T11:58:43.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=csr_pool 2026-02-17T11:58:43.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=ROBJ0 2026-02-17T11:58:43.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map csr_pool ROBJ0 2026-02-17T11:58:43.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T11:58:43.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1020: TEST_corrupt_scrub_replicated: local primary=1 2026-02-17T11:58:43.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1023: TEST_corrupt_scrub_replicated: get_asok_path osd.0 2026-02-17T11:58:43.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T11:58:43.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T11:58:43.234 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:58:43.234 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:58:43.235 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:58:43.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.0.asok 2026-02-17T11:58:43.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1023: TEST_corrupt_scrub_replicated: CEPH_ARGS= 2026-02-17T11:58:43.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1023: TEST_corrupt_scrub_replicated: ceph daemon /tmp/ceph-asok.40886/ceph-osd.0.asok config set osd_deep_scrub_update_digest_min_age 0 2026-02-17T11:58:43.297 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T11:58:43.297 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_deep_scrub_update_digest_min_age = '' (not observed, change may require restart) " 2026-02-17T11:58:43.297 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:58:43.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1025: TEST_corrupt_scrub_replicated: get_asok_path osd.1 2026-02-17T11:58:43.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T11:58:43.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T11:58:43.305 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T11:58:43.305 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:58:43.305 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:58:43.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.1.asok 2026-02-17T11:58:43.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1025: TEST_corrupt_scrub_replicated: CEPH_ARGS= 2026-02-17T11:58:43.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1025: TEST_corrupt_scrub_replicated: ceph daemon /tmp/ceph-asok.40886/ceph-osd.1.asok config set osd_deep_scrub_update_digest_min_age 0 2026-02-17T11:58:43.357 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T11:58:43.358 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_deep_scrub_update_digest_min_age = '' (not observed, change may require restart) " 2026-02-17T11:58:43.358 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T11:58:43.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1027: TEST_corrupt_scrub_replicated: pg_deep_scrub 3.0 2026-02-17T11:58:43.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: pg_deep_scrub: local pgid=3.0 2026-02-17T11:58:43.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: pg_deep_scrub: wait_for_pg_clean 3.0 2026-02-17T11:58:43.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=3.0 2026-02-17T11:58:43.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T11:58:43.365 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:58:43.365 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:58:43.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:58:43.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:58:43.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:58:43.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T11:58:43.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T11:58:43.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T11:58:43.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T11:58:43.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:58:43.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:58:43.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:58:43.806 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:58:43.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:58:43.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:58:43.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:58:43.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-17T11:58:43.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-17T11:58:43.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-17T11:58:43.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:58:43.877 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:58:43.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672968 2026-02-17T11:58:43.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672968 2026-02-17T11:58:43.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-42949672968' 2026-02-17T11:58:43.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:58:43.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-17T11:58:43.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:58:43.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:58:43.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-17T11:58:43.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:58:43.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-17T11:58:43.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-17T11:58:43.955 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836489 2026-02-17T11:58:43.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:58:44.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-17T11:58:44.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:58:45.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:58:45.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:58:45.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-17T11:58:45.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:58:46.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:58:46.274 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:58:46.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836489 2026-02-17T11:58:46.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:58:46.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672968 2026-02-17T11:58:46.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:58:46.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:58:46.430 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672968 2026-02-17T11:58:46.430 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:58:46.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672968 2026-02-17T11:58:46.432 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672968 2026-02-17T11:58:46.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672968' 2026-02-17T11:58:46.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:58:46.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672968 -lt 42949672968 2026-02-17T11:58:46.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T11:58:46.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-17T11:58:46.589 INFO:tasks.workunit.client.0.trial128.stdout:#---------- 3.0 loop 0 2026-02-17T11:58:46.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 3.0 2026-02-17T11:58:46.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=3.0 2026-02-17T11:58:46.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T11:58:46.590 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 3.0 query 2026-02-17T11:58:46.590 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T11:58:46.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T11:58:46.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T11:58:46.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T11:58:46.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T11:58:46.663 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:58:46.663 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:58:46.663 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:58:46.663 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:58:46.663 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:58:46.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: local last_scrub=2026-02-17T11:58:28.894235+0000 2026-02-17T11:58:46.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: pg_deep_scrub: ceph pg deep-scrub 3.0 2026-02-17T11:58:46.958 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2026-02-17T11:58:46.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: pg_deep_scrub: wait_for_scrub 3.0 2026-02-17T11:58:28.894235+0000 last_deep_scrub_stamp 2026-02-17T11:58:46.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T11:58:46.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:58:28.894235+0000 2026-02-17T11:58:46.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-17T11:58:46.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T11:58:46.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:58:46.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:58:46.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:58:46.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:58:46.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:58:46.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:58:47.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:58:28.894235+0000 '>' 2026-02-17T11:58:28.894235+0000 2026-02-17T11:58:47.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:58:48.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:58:48.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:58:48.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:58:48.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:58:48.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:58:48.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:58:48.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:58:48.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:58:28.894235+0000 '>' 2026-02-17T11:58:28.894235+0000 2026-02-17T11:58:48.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:58:49.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:58:49.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:58:49.280 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:58:49.280 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:58:49.280 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:58:49.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:58:49.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:58:49.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:58:28.894235+0000 '>' 2026-02-17T11:58:28.894235+0000 2026-02-17T11:58:49.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T11:58:50.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T11:58:50.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T11:58:50.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T11:58:50.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T11:58:50.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T11:58:50.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T11:58:50.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T11:58:50.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:58:47.837225+0000 '>' 2026-02-17T11:58:28.894235+0000 2026-02-17T11:58:50.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T11:58:50.590 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: seq 1 19 2026-02-17T11:58:50.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:58:50.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ1 2026-02-17T11:58:50.592 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 1 % 2 2026-02-17T11:58:50.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T11:58:50.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T11:58:50.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1038: TEST_corrupt_scrub_replicated: local payload=UVWXYZZZ 2026-02-17T11:58:50.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1039: TEST_corrupt_scrub_replicated: echo UVWXYZZZ 2026-02-17T11:58:50.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1040: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:58:50.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:58:50.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:58:50.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:58:50.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:58:50.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:58:50.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:58:50.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:58:50.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:58:50.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:58:50.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:58:50.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:58:50.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:58:50.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:58:50.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:58:50.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:58:51.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:58:51.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:58:51.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:58:51.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:58:51.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:58:51.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:58:51.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:58:51.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ1 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:58:52.559 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 3.0_head,#-5:00000000:::scrub_3.0:head#, (61) No data available 2026-02-17T11:58:53.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:58:53.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:58:53.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:58:53.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:58:53.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:58:53.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:58:53.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:58:53.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:58:53.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:58:53.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:58:53.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:58:53.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:58:53.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:58:53.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:58:53.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:58:53.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:58:53.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:58:53.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:58:53.097 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:58:53.097 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:58:53.097 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:58:53.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:58:53.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:58:53.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:58:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:58:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:58:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:58:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:58:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:58:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:58:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:58:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:58:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:58:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:58:53.099 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:58:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:58:53.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:58:53.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:58:53.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:58:53.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:58:53.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:58:53.120 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:53.113+0000 7f6e8ac198c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:58:53.121 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:53.113+0000 7f6e8ac198c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:58:53.122 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:53.113+0000 7f6e8ac198c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:58:53.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:58:53.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:58:53.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:58:53.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:58:53.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:58:53.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:53.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:58:53.255 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:58:53.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:53.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:58:53.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:58:54.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:58:54.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:54.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:58:54.406 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:58:54.406 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:54.397+0000 7f6e8ac198c0 -1 Falling back to public interface 2026-02-17T11:58:54.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:54.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:58:54.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:58:55.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:58:55.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:55.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:58:55.562 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:58:55.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:55.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:58:55.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:58:56.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:58:56.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:56.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:58:56.715 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:58:56.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:56.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:58:56.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:58:57.401 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:58:57.393+0000 7f6e8ac198c0 -1 osd.1 62 log_to_monitors true 2026-02-17T11:58:57.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:58:57.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:58:57.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:58:57.867 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:58:57.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:58:57.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:58:58.016 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 66 up_thru 66 down_at 63 last_clean_interval [10,62) [v2:127.0.0.1:6810/3102014808,v1:127.0.0.1:6811/3102014808] [v2:127.0.0.1:6812/3102014808,v1:127.0.0.1:6813/3102014808] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T11:58:58.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:58:58.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:58:58.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:58:58.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:58:58.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:58:58.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:58:58.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:58:58.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:58:58.017 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:58:58.018 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:58:58.019 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:58:58.019 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:58:58.019 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:58:58.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:58:58.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:58:58.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:58:58.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:58:58.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:58:58.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:58:58.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:58:58.282 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:58:58.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:58:58.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:58:58.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:58:58.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836493 2026-02-17T11:58:58.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836493 2026-02-17T11:58:58.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493' 2026-02-17T11:58:58.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:58:58.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:58:58.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=283467841538 2026-02-17T11:58:58.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 283467841538 2026-02-17T11:58:58.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493 1-283467841538' 2026-02-17T11:58:58.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:58:58.430 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836493 2026-02-17T11:58:58.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:58:58.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:58:58.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836493 2026-02-17T11:58:58.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:58:58.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836493 2026-02-17T11:58:58.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-17T11:58:58.434 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836493 2026-02-17T11:58:58.435 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:58:58.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836493 2026-02-17T11:58:58.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:58:59.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:58:59.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:58:59.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836493 2026-02-17T11:58:59.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:59:00.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T11:59:00.754 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:59:00.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836494 -lt 21474836493 2026-02-17T11:59:00.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:59:00.907 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-283467841538 2026-02-17T11:59:00.907 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:59:00.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:59:00.909 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-283467841538 2026-02-17T11:59:00.909 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:59:00.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=283467841538 2026-02-17T11:59:00.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 283467841538' 2026-02-17T11:59:00.911 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 283467841538 2026-02-17T11:59:00.911 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:59:01.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 283467841538 -lt 283467841538 2026-02-17T11:59:01.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:59:01.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:59:01.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:59:01.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:59:01.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:59:01.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:59:01.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:59:01.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:59:01.253 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:59:01.253 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:59:01.253 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:59:01.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:59:01.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:59:01.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:59:01.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:59:01.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:59:01.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:59:01.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:59:01.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:59:01.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ2 2026-02-17T11:59:01.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 2 % 2 2026-02-17T11:59:01.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T11:59:01.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T11:59:01.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1045: TEST_corrupt_scrub_replicated: local payload=UVWXYZ 2026-02-17T11:59:01.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1046: TEST_corrupt_scrub_replicated: echo UVWXYZ 2026-02-17T11:59:01.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1047: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:59:01.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:59:01.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:59:01.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:59:01.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:59:01.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:59:01.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:59:01.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:59:01.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:59:01.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:59:01.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:59:01.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:59:01.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:59:01.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:59:01.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:59:01.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:59:02.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:59:02.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:59:02.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:59:02.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:59:02.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:59:02.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:59:02.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:59:02.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ2 set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T11:59:04.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:59:04.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:59:04.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:59:04.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:59:04.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:59:04.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:59:04.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:59:04.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:59:04.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:59:04.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:59:04.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:59:04.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:59:04.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:59:04.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:59:04.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:59:04.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:59:04.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:59:04.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:59:04.109 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:59:04.109 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:59:04.109 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:59:04.111 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:59:04.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:59:04.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:59:04.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:59:04.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:59:04.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:59:04.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:59:04.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:59:04.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:59:04.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:59:04.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:59:04.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:59:04.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:59:04.112 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:59:04.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:59:04.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:59:04.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:59:04.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:59:04.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:59:04.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:59:04.132 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:04.125+0000 7f1bf2ccc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:04.134 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:04.125+0000 7f1bf2ccc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:04.135 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:04.129+0000 7f1bf2ccc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:04.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:59:04.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:59:04.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:59:04.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:59:04.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:59:04.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:04.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:59:04.269 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:59:04.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:04.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:59:04.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:05.373 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:05.365+0000 7f1bf2ccc8c0 -1 Falling back to public interface 2026-02-17T11:59:05.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:05.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:05.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:59:05.424 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:59:05.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:05.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:59:05.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:06.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:06.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:06.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:59:06.580 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:59:06.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:06.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:59:06.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:07.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:07.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:07.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:59:07.737 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:59:07.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:07.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:59:07.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:08.352 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:08.345+0000 7f1bf2ccc8c0 -1 osd.0 68 log_to_monitors true 2026-02-17T11:59:08.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:08.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:08.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:59:08.894 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:59:08.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:08.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:59:09.048 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 72 up_thru 72 down_at 69 last_clean_interval [5,68) [v2:127.0.0.1:6802/2667938346,v1:127.0.0.1:6803/2667938346] [v2:127.0.0.1:6804/2667938346,v1:127.0.0.1:6805/2667938346] exists,up 298610f2-1d9b-4f6c-b9da-00d205e834ad 2026-02-17T11:59:09.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:59:09.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:59:09.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:59:09.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:59:09.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:59:09.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:59:09.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:59:09.049 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:59:09.050 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:59:09.050 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:59:09.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:59:09.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:59:09.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:59:09.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:59:09.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:59:09.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:59:09.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:59:09.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:59:09.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:59:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:59:09.320 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:59:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:59:09.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:59:09.321 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:59:09.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=309237645314 2026-02-17T11:59:09.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 309237645314 2026-02-17T11:59:09.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-309237645314' 2026-02-17T11:59:09.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:59:09.396 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:59:09.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=283467841541 2026-02-17T11:59:09.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 283467841541 2026-02-17T11:59:09.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-309237645314 1-283467841541' 2026-02-17T11:59:09.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:59:09.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-309237645314 2026-02-17T11:59:09.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:59:09.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:59:09.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-309237645314 2026-02-17T11:59:09.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:59:09.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=309237645314 2026-02-17T11:59:09.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 309237645314' 2026-02-17T11:59:09.470 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 309237645314 2026-02-17T11:59:09.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:59:09.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836494 -lt 309237645314 2026-02-17T11:59:09.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:59:10.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:59:10.636 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:59:10.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 309237645314 -lt 309237645314 2026-02-17T11:59:10.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:59:10.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-283467841541 2026-02-17T11:59:10.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:59:10.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:59:10.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-283467841541 2026-02-17T11:59:10.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:59:10.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=283467841541 2026-02-17T11:59:10.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 283467841541' 2026-02-17T11:59:10.792 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 283467841541 2026-02-17T11:59:10.793 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:59:10.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 283467841541 -lt 283467841541 2026-02-17T11:59:10.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:59:10.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:59:10.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:59:11.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:59:11.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:59:11.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:59:11.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:59:11.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:59:11.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:59:11.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:59:11.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:59:11.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:59:11.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:59:11.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:59:11.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:59:11.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:59:11.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:59:11.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:59:11.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:59:11.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ3 2026-02-17T11:59:11.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 3 % 2 2026-02-17T11:59:11.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T11:59:11.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T11:59:11.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1052: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ3 remove 2026-02-17T11:59:11.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:59:11.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:59:11.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:59:11.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:59:11.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ3 remove 2026-02-17T11:59:11.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:59:11.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:59:11.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:59:11.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:59:11.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:59:11.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:59:11.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:59:11.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:59:11.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:59:11.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:59:12.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:59:12.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ3 remove 2026-02-17T11:59:12.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:59:12.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:59:12.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:59:12.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:59:12.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:59:12.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ3 remove 2026-02-17T11:59:13.440 INFO:tasks.workunit.client.0.trial128.stdout:remove #3:f2a5b2a4:::ROBJ3:head# 2026-02-17T11:59:13.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:59:13.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:59:13.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:59:13.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:59:13.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:59:13.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:59:13.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:59:13.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:59:13.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:59:13.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:59:13.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:59:13.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:59:13.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:59:13.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:59:13.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:59:13.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:59:13.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:59:13.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:59:13.996 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:59:13.996 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:59:13.997 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:59:13.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:59:13.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:59:13.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:59:13.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:59:13.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:59:14.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:59:14.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:59:14.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:59:14.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:59:14.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:59:14.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:59:14.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:59:14.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:59:14.000 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:59:14.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:59:14.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:59:14.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:59:14.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:59:14.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:59:14.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:59:14.015 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:14.005+0000 7f66a7edb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:14.016 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:14.009+0000 7f66a7edb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:14.017 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:14.009+0000 7f66a7edb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:14.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:59:14.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:59:14.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:59:14.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:59:14.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:59:14.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:14.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:59:14.153 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:59:14.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:14.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:59:14.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:15.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:15.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:15.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:59:15.310 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:59:15.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:15.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:59:15.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:15.501 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:15.493+0000 7f66a7edb8c0 -1 Falling back to public interface 2026-02-17T11:59:16.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:16.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:16.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:59:16.464 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:59:16.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:16.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:59:16.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:17.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:17.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:17.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:59:17.613 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:59:17.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:17.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:59:17.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:18.501 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:18.493+0000 7f66a7edb8c0 -1 osd.1 73 log_to_monitors true 2026-02-17T11:59:18.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:18.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:18.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:59:18.768 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:59:18.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:18.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:59:18.916 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 77 up_thru 77 down_at 74 last_clean_interval [66,73) [v2:127.0.0.1:6810/2004271244,v1:127.0.0.1:6811/2004271244] [v2:127.0.0.1:6812/2004271244,v1:127.0.0.1:6813/2004271244] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T11:59:18.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:59:18.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:59:18.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:59:18.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:59:18.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:59:18.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:59:18.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:59:18.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:59:18.918 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:59:18.918 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:59:18.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:59:18.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:59:18.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:59:19.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:59:19.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:59:19.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:59:19.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:59:19.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:59:19.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:59:19.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:59:19.199 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:59:19.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:59:19.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:59:19.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:59:19.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=309237645317 2026-02-17T11:59:19.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 309237645317 2026-02-17T11:59:19.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-309237645317' 2026-02-17T11:59:19.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:59:19.269 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:59:19.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=330712481794 2026-02-17T11:59:19.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 330712481794 2026-02-17T11:59:19.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-309237645317 1-330712481794' 2026-02-17T11:59:19.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:59:19.342 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-309237645317 2026-02-17T11:59:19.342 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:59:19.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:59:19.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-309237645317 2026-02-17T11:59:19.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:59:19.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=309237645317 2026-02-17T11:59:19.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 309237645317' 2026-02-17T11:59:19.346 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 309237645317 2026-02-17T11:59:19.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:59:19.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 309237645315 -lt 309237645317 2026-02-17T11:59:19.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:59:20.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:59:20.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:59:20.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 309237645317 -lt 309237645317 2026-02-17T11:59:20.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:59:20.666 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-330712481794 2026-02-17T11:59:20.666 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:59:20.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:59:20.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-330712481794 2026-02-17T11:59:20.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:59:20.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=330712481794 2026-02-17T11:59:20.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 330712481794' 2026-02-17T11:59:20.670 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 330712481794 2026-02-17T11:59:20.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:59:20.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 330712481794 -lt 330712481794 2026-02-17T11:59:20.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:59:20.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:59:20.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:59:21.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:59:21.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:59:21.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:59:21.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:59:21.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:59:21.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:59:21.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:59:21.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:59:21.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:59:21.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:59:21.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:59:21.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:59:21.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:59:21.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:59:21.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:59:21.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:59:21.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ4 2026-02-17T11:59:21.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 4 % 2 2026-02-17T11:59:21.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T11:59:21.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T11:59:21.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1057: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ4 set-omap key-ROBJ4 td/osd-scrub-repair/CORRUPT 2026-02-17T11:59:21.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:59:21.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:59:21.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:59:21.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:59:21.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ4 set-omap key-ROBJ4 td/osd-scrub-repair/CORRUPT 2026-02-17T11:59:21.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:59:21.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:59:21.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:59:21.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:59:21.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:59:21.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:59:21.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:59:21.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:59:21.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:59:21.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:59:22.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:59:22.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ4 set-omap key-ROBJ4 td/osd-scrub-repair/CORRUPT 2026-02-17T11:59:22.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:59:22.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:59:22.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:59:22.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:59:22.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:59:22.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ4 set-omap key-ROBJ4 td/osd-scrub-repair/CORRUPT 2026-02-17T11:59:23.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:59:23.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:59:23.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:59:23.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:59:23.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:59:23.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:59:23.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:59:23.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:59:23.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:59:23.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:59:23.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:59:23.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:59:23.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:59:23.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:59:23.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:59:23.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:59:23.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:59:23.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:59:23.817 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:59:23.817 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:59:23.817 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:59:23.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:59:23.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:59:23.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:59:23.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:59:23.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:59:23.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:59:23.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:59:23.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:59:23.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:59:23.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:59:23.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:59:23.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:59:23.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:59:23.820 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:59:23.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:59:23.821 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:59:23.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:59:23.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:59:23.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:59:23.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:59:23.840 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:23.833+0000 7f8237a6a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:23.841 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:23.833+0000 7f8237a6a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:23.842 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:23.833+0000 7f8237a6a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:23.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:59:23.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:59:23.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:59:23.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:59:23.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:59:23.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:23.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:59:23.971 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:59:23.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:23.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:59:24.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:25.073 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:25.065+0000 7f8237a6a8c0 -1 Falling back to public interface 2026-02-17T11:59:25.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:25.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:25.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:59:25.126 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:59:25.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:25.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:59:25.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:26.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:26.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:26.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:59:26.277 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:59:26.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:26.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:59:26.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:27.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:27.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:27.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:59:27.435 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:59:27.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:27.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:59:27.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:28.069 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:28.061+0000 7f8237a6a8c0 -1 osd.0 78 log_to_monitors true 2026-02-17T11:59:28.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:28.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:28.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:59:28.591 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:59:28.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:28.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:59:28.745 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 82 up_thru 82 down_at 79 last_clean_interval [72,78) [v2:127.0.0.1:6802/3996662973,v1:127.0.0.1:6803/3996662973] [v2:127.0.0.1:6804/3996662973,v1:127.0.0.1:6805/3996662973] exists,up 298610f2-1d9b-4f6c-b9da-00d205e834ad 2026-02-17T11:59:28.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:59:28.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:59:28.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:59:28.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:59:28.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:59:28.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:59:28.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:59:28.746 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:59:28.746 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:59:28.746 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:59:28.746 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:59:28.746 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:59:28.746 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:59:28.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:59:28.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:59:28.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:59:28.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:59:28.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:59:28.861 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:59:29.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:59:29.016 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:59:29.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:59:29.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:59:29.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:59:29.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=352187318274 2026-02-17T11:59:29.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 352187318274 2026-02-17T11:59:29.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318274' 2026-02-17T11:59:29.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:59:29.089 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:59:29.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=330712481797 2026-02-17T11:59:29.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 330712481797 2026-02-17T11:59:29.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318274 1-330712481797' 2026-02-17T11:59:29.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:59:29.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-352187318274 2026-02-17T11:59:29.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:59:29.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:59:29.163 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-352187318274 2026-02-17T11:59:29.163 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:59:29.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=352187318274 2026-02-17T11:59:29.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 352187318274' 2026-02-17T11:59:29.165 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 352187318274 2026-02-17T11:59:29.166 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:59:29.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 309237645317 -lt 352187318274 2026-02-17T11:59:29.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:59:30.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:59:30.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:59:30.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318274 -lt 352187318274 2026-02-17T11:59:30.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:59:30.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-330712481797 2026-02-17T11:59:30.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:59:30.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:59:30.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-330712481797 2026-02-17T11:59:30.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:59:30.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=330712481797 2026-02-17T11:59:30.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 330712481797' 2026-02-17T11:59:30.488 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 330712481797 2026-02-17T11:59:30.488 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:59:30.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 330712481797 -lt 330712481797 2026-02-17T11:59:30.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:59:30.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:59:30.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:59:30.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:59:30.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:59:30.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:59:30.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:59:30.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:59:30.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:59:30.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:59:30.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:59:30.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:59:30.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:59:30.972 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:59:30.972 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:59:31.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:59:31.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:59:31.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:59:31.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:59:31.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ5 2026-02-17T11:59:31.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 5 % 2 2026-02-17T11:59:31.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T11:59:31.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T11:59:31.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1062: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ5 rm-omap key-ROBJ5 2026-02-17T11:59:31.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:59:31.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:59:31.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:59:31.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:59:31.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ5 rm-omap key-ROBJ5 2026-02-17T11:59:31.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:59:31.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:59:31.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:59:31.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:59:31.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:59:31.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:59:31.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:59:31.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:59:31.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:59:31.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:59:32.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:59:32.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ5 rm-omap key-ROBJ5 2026-02-17T11:59:32.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:59:32.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:59:32.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:59:32.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:59:32.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:59:32.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ5 rm-omap key-ROBJ5 2026-02-17T11:59:33.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:59:33.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:59:33.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:59:33.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:59:33.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:59:33.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:59:33.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:59:33.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:59:33.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:59:33.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:59:33.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:59:33.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:59:33.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:59:33.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:59:33.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:59:33.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:59:33.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:59:33.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:59:33.641 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:59:33.641 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:59:33.641 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:59:33.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:59:33.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:59:33.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:59:33.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:59:33.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:59:33.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:59:33.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:59:33.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:59:33.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:59:33.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:59:33.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:59:33.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:59:33.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:59:33.643 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:59:33.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:59:33.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:59:33.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:59:33.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:59:33.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:59:33.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:59:33.663 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:33.657+0000 7fa706a6f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:33.665 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:33.657+0000 7fa706a6f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:33.666 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:33.657+0000 7fa706a6f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:33.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:59:33.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:59:33.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:59:33.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:59:33.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:59:33.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:33.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:59:33.792 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:59:33.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:33.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:59:33.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:34.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:34.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:34.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:59:34.953 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:59:34.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:34.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:59:35.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:35.148 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:35.141+0000 7fa706a6f8c0 -1 Falling back to public interface 2026-02-17T11:59:36.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:36.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:36.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:59:36.110 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:59:36.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:36.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:59:36.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:37.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:37.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:37.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:59:37.265 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:59:37.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:37.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:59:37.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:38.101 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:38.093+0000 7fa706a6f8c0 -1 osd.1 84 log_to_monitors true 2026-02-17T11:59:38.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:38.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:38.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:59:38.418 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:59:38.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:38.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:59:38.573 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 88 up_thru 88 down_at 85 last_clean_interval [77,84) [v2:127.0.0.1:6810/4216803543,v1:127.0.0.1:6811/4216803543] [v2:127.0.0.1:6812/4216803543,v1:127.0.0.1:6813/4216803543] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T11:59:38.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:59:38.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:59:38.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:59:38.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:59:38.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:59:38.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:59:38.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:59:38.574 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:59:38.575 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:59:38.576 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:59:38.576 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:59:38.576 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:59:38.576 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:59:38.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:59:38.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:59:38.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:59:38.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:59:38.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:59:38.691 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:59:38.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:59:38.848 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:59:38.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:59:38.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:59:38.849 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:59:38.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=352187318277 2026-02-17T11:59:38.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 352187318277 2026-02-17T11:59:38.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318277' 2026-02-17T11:59:38.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:59:38.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:59:38.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=377957122050 2026-02-17T11:59:38.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 377957122050 2026-02-17T11:59:38.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-352187318277 1-377957122050' 2026-02-17T11:59:38.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:59:38.990 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-352187318277 2026-02-17T11:59:38.990 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:59:38.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:59:38.992 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-352187318277 2026-02-17T11:59:38.992 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:59:38.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=352187318277 2026-02-17T11:59:38.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 352187318277' 2026-02-17T11:59:38.994 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 352187318277 2026-02-17T11:59:38.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:59:39.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318275 -lt 352187318277 2026-02-17T11:59:39.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:59:40.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:59:40.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:59:40.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318277 -lt 352187318277 2026-02-17T11:59:40.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:59:40.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-377957122050 2026-02-17T11:59:40.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:59:40.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:59:40.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-377957122050 2026-02-17T11:59:40.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:59:40.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=377957122050 2026-02-17T11:59:40.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 377957122050' 2026-02-17T11:59:40.312 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 377957122050 2026-02-17T11:59:40.312 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:59:40.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 377957122050 -lt 377957122050 2026-02-17T11:59:40.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:59:40.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:59:40.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:59:40.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:59:40.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:59:40.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:59:40.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:59:40.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:59:40.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:59:40.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:59:40.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:59:40.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:59:40.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:59:40.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:59:40.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:59:40.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:59:40.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:59:40.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:59:40.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:59:40.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ6 2026-02-17T11:59:40.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 6 % 2 2026-02-17T11:59:40.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T11:59:40.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T11:59:40.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1067: TEST_corrupt_scrub_replicated: echo extra 2026-02-17T11:59:40.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1068: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ6 set-omap key2-ROBJ6 td/osd-scrub-repair/extra-val 2026-02-17T11:59:40.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:59:40.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:59:40.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T11:59:40.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:59:40.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ6 set-omap key2-ROBJ6 td/osd-scrub-repair/extra-val 2026-02-17T11:59:40.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:59:40.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:59:40.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T11:59:40.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:59:40.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T11:59:40.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:59:40.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:59:40.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:59:40.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:59:40.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:59:42.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:59:42.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ6 set-omap key2-ROBJ6 td/osd-scrub-repair/extra-val 2026-02-17T11:59:42.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:59:42.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:59:42.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T11:59:42.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:59:42.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:59:42.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ6 set-omap key2-ROBJ6 td/osd-scrub-repair/extra-val 2026-02-17T11:59:43.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T11:59:43.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:59:43.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:59:43.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T11:59:43.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:59:43.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T11:59:43.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:59:43.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:59:43.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:59:43.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:59:43.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T11:59:43.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T11:59:43.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:59:43.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:59:43.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:59:43.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:59:43.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:59:43.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:59:43.469 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:59:43.469 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:59:43.470 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:59:43.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:59:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:59:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:59:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:59:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:59:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:59:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:59:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:59:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:59:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:59:43.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:59:43.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T11:59:43.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T11:59:43.471 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T11:59:43.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:59:43.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T11:59:43.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T11:59:43.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:59:43.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:59:43.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:59:43.491 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:43.485+0000 7f187213e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:43.493 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:43.485+0000 7f187213e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:43.494 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:43.485+0000 7f187213e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:43.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T11:59:43.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:59:43.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T11:59:43.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:59:43.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:59:43.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:43.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:59:43.619 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:59:43.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:43.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:59:43.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:44.761 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:44.753+0000 7f187213e8c0 -1 Falling back to public interface 2026-02-17T11:59:44.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:44.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:44.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:59:44.778 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:59:44.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:44.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:59:44.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:45.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:45.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:45.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:59:45.935 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:59:45.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:45.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:59:46.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:47.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:47.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:47.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:59:47.093 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:59:47.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:47.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:59:47.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:47.781 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:47.773+0000 7f187213e8c0 -1 osd.0 89 log_to_monitors true 2026-02-17T11:59:48.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:48.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:48.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:59:48.248 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:59:48.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:48.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T11:59:48.401 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 93 up_thru 93 down_at 90 last_clean_interval [82,89) [v2:127.0.0.1:6802/2191988634,v1:127.0.0.1:6803/2191988634] [v2:127.0.0.1:6804/2191988634,v1:127.0.0.1:6805/2191988634] exists,up 298610f2-1d9b-4f6c-b9da-00d205e834ad 2026-02-17T11:59:48.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:59:48.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:59:48.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:59:48.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:59:48.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:59:48.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:59:48.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:59:48.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:59:48.403 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:59:48.403 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:59:48.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:59:48.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:59:48.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:59:48.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:59:48.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:59:48.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:59:48.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:59:48.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:59:48.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:59:48.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:59:48.676 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:59:48.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:59:48.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:59:48.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:59:48.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=399431958530 2026-02-17T11:59:48.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 399431958530 2026-02-17T11:59:48.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-399431958530' 2026-02-17T11:59:48.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:59:48.747 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:59:48.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=377957122053 2026-02-17T11:59:48.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 377957122053 2026-02-17T11:59:48.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-399431958530 1-377957122053' 2026-02-17T11:59:48.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:59:48.818 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-399431958530 2026-02-17T11:59:48.818 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:59:48.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:59:48.820 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-399431958530 2026-02-17T11:59:48.820 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:59:48.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=399431958530 2026-02-17T11:59:48.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 399431958530' 2026-02-17T11:59:48.822 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 399431958530 2026-02-17T11:59:48.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:59:48.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 352187318277 -lt 399431958530 2026-02-17T11:59:48.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:59:49.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:59:49.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:59:50.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 399431958530 -lt 399431958530 2026-02-17T11:59:50.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:59:50.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-377957122053 2026-02-17T11:59:50.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:59:50.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:59:50.144 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-377957122053 2026-02-17T11:59:50.144 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:59:50.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=377957122053 2026-02-17T11:59:50.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 377957122053' 2026-02-17T11:59:50.145 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 377957122053 2026-02-17T11:59:50.146 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T11:59:50.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 377957122053 -lt 377957122053 2026-02-17T11:59:50.302 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T11:59:50.302 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:59:50.303 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:59:50.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T11:59:50.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T11:59:50.489 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T11:59:50.489 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T11:59:50.490 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T11:59:50.490 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T11:59:50.490 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T11:59:50.490 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T11:59:50.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T11:59:50.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T11:59:50.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T11:59:50.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T11:59:50.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T11:59:50.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T11:59:50.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T11:59:50.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1069: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/extra-val 2026-02-17T11:59:50.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T11:59:50.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ7 2026-02-17T11:59:50.834 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 7 % 2 2026-02-17T11:59:50.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T11:59:50.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T11:59:50.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1074: TEST_corrupt_scrub_replicated: echo -n newheader 2026-02-17T11:59:50.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1075: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ7 set-omaphdr td/osd-scrub-repair/hdr 2026-02-17T11:59:50.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T11:59:50.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T11:59:50.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T11:59:50.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T11:59:50.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ7 set-omaphdr td/osd-scrub-repair/hdr 2026-02-17T11:59:50.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T11:59:50.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T11:59:50.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T11:59:50.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T11:59:50.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T11:59:50.837 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T11:59:50.838 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T11:59:50.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T11:59:50.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T11:59:50.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T11:59:52.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T11:59:52.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ7 set-omaphdr td/osd-scrub-repair/hdr 2026-02-17T11:59:52.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T11:59:52.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T11:59:52.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T11:59:52.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T11:59:52.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:59:52.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ7 set-omaphdr td/osd-scrub-repair/hdr 2026-02-17T11:59:53.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T11:59:53.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T11:59:53.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T11:59:53.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T11:59:53.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T11:59:53.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T11:59:53.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T11:59:53.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T11:59:53.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T11:59:53.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T11:59:53.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T11:59:53.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T11:59:53.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T11:59:53.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T11:59:53.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T11:59:53.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T11:59:53.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T11:59:53.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T11:59:53.320 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T11:59:53.320 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T11:59:53.320 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T11:59:53.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:59:53.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T11:59:53.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T11:59:53.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T11:59:53.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T11:59:53.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T11:59:53.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T11:59:53.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T11:59:53.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T11:59:53.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T11:59:53.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T11:59:53.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T11:59:53.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T11:59:53.324 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T11:59:53.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T11:59:53.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T11:59:53.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T11:59:53.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T11:59:53.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T11:59:53.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T11:59:53.343 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:53.337+0000 7f1e5a5868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:53.344 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:53.337+0000 7f1e5a5868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:53.345 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:53.337+0000 7f1e5a5868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T11:59:53.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T11:59:53.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T11:59:53.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T11:59:53.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T11:59:53.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T11:59:53.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:53.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T11:59:53.478 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T11:59:53.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:53.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:59:53.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:54.584 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:54.577+0000 7f1e5a5868c0 -1 Falling back to public interface 2026-02-17T11:59:54.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:54.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:54.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T11:59:54.631 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T11:59:54.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:54.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:59:54.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:55.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:55.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:55.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T11:59:55.783 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T11:59:55.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:55.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:59:55.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:56.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:56.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:56.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T11:59:56.937 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T11:59:56.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:56.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:59:57.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T11:59:57.563 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T11:59:57.557+0000 7f1e5a5868c0 -1 osd.1 95 log_to_monitors true 2026-02-17T11:59:58.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T11:59:58.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T11:59:58.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T11:59:58.087 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T11:59:58.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T11:59:58.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T11:59:58.236 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 99 up_thru 99 down_at 96 last_clean_interval [88,95) [v2:127.0.0.1:6810/1842731815,v1:127.0.0.1:6811/1842731815] [v2:127.0.0.1:6812/1842731815,v1:127.0.0.1:6813/1842731815] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T11:59:58.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T11:59:58.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T11:59:58.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T11:59:58.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T11:59:58.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T11:59:58.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T11:59:58.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T11:59:58.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T11:59:58.238 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T11:59:58.238 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T11:59:58.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T11:59:58.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T11:59:58.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T11:59:58.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T11:59:58.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T11:59:58.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T11:59:58.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T11:59:58.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T11:59:58.353 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T11:59:58.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T11:59:58.506 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T11:59:58.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T11:59:58.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:59:58.507 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T11:59:58.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=399431958533 2026-02-17T11:59:58.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 399431958533 2026-02-17T11:59:58.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-399431958533' 2026-02-17T11:59:58.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T11:59:58.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T11:59:58.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=425201762306 2026-02-17T11:59:58.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 425201762306 2026-02-17T11:59:58.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-399431958533 1-425201762306' 2026-02-17T11:59:58.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:59:58.653 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-399431958533 2026-02-17T11:59:58.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:59:58.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T11:59:58.655 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-399431958533 2026-02-17T11:59:58.655 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:59:58.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=399431958533 2026-02-17T11:59:58.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 399431958533' 2026-02-17T11:59:58.657 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 399431958533 2026-02-17T11:59:58.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:59:58.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 399431958531 -lt 399431958533 2026-02-17T11:59:58.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T11:59:59.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T11:59:59.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T11:59:59.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 399431958533 -lt 399431958533 2026-02-17T11:59:59.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T11:59:59.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-425201762306 2026-02-17T11:59:59.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T11:59:59.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T11:59:59.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-425201762306 2026-02-17T11:59:59.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T11:59:59.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=425201762306 2026-02-17T11:59:59.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 425201762306' 2026-02-17T11:59:59.982 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 425201762306 2026-02-17T11:59:59.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:00:00.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762306 -lt 425201762306 2026-02-17T12:00:00.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:00:00.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:00:00.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:00:00.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:00:00.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:00:00.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:00:00.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:00:00.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:00:00.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:00:00.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:00:00.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:00:00.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:00:00.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:00:00.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:00:00.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:00:00.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:00:00.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:00:00.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:00:00.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1076: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/hdr 2026-02-17T12:00:00.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T12:00:00.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ8 2026-02-17T12:00:00.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 8 % 2 2026-02-17T12:00:00.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T12:00:00.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T12:00:00.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1080: TEST_corrupt_scrub_replicated: rados --pool csr_pool setxattr ROBJ8 key1-ROBJ8 val1-ROBJ8 2026-02-17T12:00:00.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1081: TEST_corrupt_scrub_replicated: rados --pool csr_pool setxattr ROBJ8 key2-ROBJ8 val2-ROBJ8 2026-02-17T12:00:00.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1084: TEST_corrupt_scrub_replicated: echo -n bad-val 2026-02-17T12:00:00.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1085: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ8 set-attr _key1-ROBJ8 td/osd-scrub-repair/bad-val 2026-02-17T12:00:00.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:00:00.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:00:00.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T12:00:00.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:00:00.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ8 set-attr _key1-ROBJ8 td/osd-scrub-repair/bad-val 2026-02-17T12:00:00.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:00:00.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:00:00.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T12:00:00.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:00:00.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T12:00:00.722 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:00:00.722 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:00:00.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:00:00.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:00:00.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:00:02.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:00:02.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ8 set-attr _key1-ROBJ8 td/osd-scrub-repair/bad-val 2026-02-17T12:00:02.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:00:02.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:00:02.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:00:02.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:00:02.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:00:02.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ8 set-attr _key1-ROBJ8 td/osd-scrub-repair/bad-val 2026-02-17T12:00:03.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T12:00:03.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:00:03.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:00:03.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:00:03.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:00:03.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:00:03.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:00:03.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:00:03.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:00:03.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:00:03.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:00:03.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:00:03.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:00:03.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:00:03.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:00:03.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:00:03.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:00:03.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:00:03.207 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:00:03.207 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:00:03.207 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:00:03.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:00:03.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:00:03.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:00:03.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:00:03.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:00:03.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:00:03.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:00:03.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:00:03.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:00:03.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:00:03.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:00:03.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:00:03.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:00:03.208 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T12:00:03.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:00:03.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T12:00:03.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:00:03.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:00:03.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:00:03.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:00:03.230 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:03.225+0000 7f84ec46e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:03.232 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:03.225+0000 7f84ec46e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:03.233 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:03.225+0000 7f84ec46e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:03.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:00:03.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:00:03.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:00:03.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:00:03.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:00:03.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:03.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:00:03.367 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:00:03.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:03.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:00:03.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:04.468 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:04.461+0000 7f84ec46e8c0 -1 Falling back to public interface 2026-02-17T12:00:04.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:04.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:04.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:00:04.517 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:00:04.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:04.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:00:04.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:05.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:05.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:05.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:00:05.666 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:00:05.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:05.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:00:05.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:06.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:06.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:06.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:00:06.818 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:00:06.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:06.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:00:06.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:07.448 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:07.441+0000 7f84ec46e8c0 -1 osd.0 101 log_to_monitors true 2026-02-17T12:00:07.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:07.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:07.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:00:07.967 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:00:07.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:07.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:00:08.114 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 105 up_thru 105 down_at 102 last_clean_interval [93,101) [v2:127.0.0.1:6802/3364356267,v1:127.0.0.1:6803/3364356267] [v2:127.0.0.1:6804/3364356267,v1:127.0.0.1:6805/3364356267] exists,up 298610f2-1d9b-4f6c-b9da-00d205e834ad 2026-02-17T12:00:08.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:00:08.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:00:08.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:00:08.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:00:08.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:00:08.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:00:08.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:00:08.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:00:08.116 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:00:08.116 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:00:08.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:00:08.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:00:08.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:00:08.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:00:08.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:00:08.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:00:08.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:00:08.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:00:08.225 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:00:08.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:00:08.382 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:00:08.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:00:08.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:00:08.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:00:08.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=450971566082 2026-02-17T12:00:08.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 450971566082 2026-02-17T12:00:08.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566082' 2026-02-17T12:00:08.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:00:08.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:00:08.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=425201762309 2026-02-17T12:00:08.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 425201762309 2026-02-17T12:00:08.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566082 1-425201762309' 2026-02-17T12:00:08.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:00:08.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-450971566082 2026-02-17T12:00:08.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:00:08.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:00:08.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-450971566082 2026-02-17T12:00:08.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:00:08.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=450971566082 2026-02-17T12:00:08.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 450971566082' 2026-02-17T12:00:08.530 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 450971566082 2026-02-17T12:00:08.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:00:08.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 450971566082 2026-02-17T12:00:08.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:00:09.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:00:09.690 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:00:09.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 450971566082 2026-02-17T12:00:09.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:00:10.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:00:10.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:00:10.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566082 -lt 450971566082 2026-02-17T12:00:10.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:00:10.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-425201762309 2026-02-17T12:00:10.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:00:10.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:00:11.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-425201762309 2026-02-17T12:00:11.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:00:11.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=425201762309 2026-02-17T12:00:11.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 425201762309' 2026-02-17T12:00:11.002 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 425201762309 2026-02-17T12:00:11.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:00:11.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 425201762309 -lt 425201762309 2026-02-17T12:00:11.165 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:00:11.166 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:00:11.166 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:00:11.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:00:11.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:00:11.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:00:11.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:00:11.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:00:11.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:00:11.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:00:11.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:00:11.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:00:11.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:00:11.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:00:11.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:00:11.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:00:11.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:00:11.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:00:11.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1086: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ8 rm-attr _key2-ROBJ8 2026-02-17T12:00:11.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:00:11.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:00:11.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:00:11.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:00:11.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ8 rm-attr _key2-ROBJ8 2026-02-17T12:00:11.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:00:11.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:00:11.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:00:11.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:00:11.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:00:11.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:00:11.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:00:11.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:00:11.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:00:11.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:00:13.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:00:13.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ8 rm-attr _key2-ROBJ8 2026-02-17T12:00:13.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:00:13.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:00:13.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:00:13.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:00:13.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:00:13.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ8 rm-attr _key2-ROBJ8 2026-02-17T12:00:14.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T12:00:14.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:00:14.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:00:14.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:00:14.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:00:14.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:00:14.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:00:14.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:00:14.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:00:14.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:00:14.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:00:14.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:00:14.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:00:14.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:00:14.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:00:14.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:00:14.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:00:14.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:00:14.181 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:00:14.181 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:00:14.181 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:00:14.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:00:14.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:00:14.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:00:14.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:00:14.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:00:14.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:00:14.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:00:14.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:00:14.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:00:14.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:00:14.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:00:14.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:00:14.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:00:14.183 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:00:14.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:00:14.183 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:00:14.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:00:14.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:00:14.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:00:14.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:00:14.203 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:14.197+0000 7f28097888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:14.204 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:14.197+0000 7f28097888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:14.206 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:14.197+0000 7f28097888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:14.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:00:14.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:00:14.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:00:14.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:00:14.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:00:14.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:14.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:00:14.332 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:00:14.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:14.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:14.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:15.448 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:15.441+0000 7f28097888c0 -1 Falling back to public interface 2026-02-17T12:00:15.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:15.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:15.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:00:15.490 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:00:15.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:15.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:15.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:16.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:16.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:16.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:00:16.642 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:00:16.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:16.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:16.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:17.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:17.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:17.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:00:17.798 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:00:17.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:17.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:17.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:18.422 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:18.413+0000 7f28097888c0 -1 osd.1 106 log_to_monitors true 2026-02-17T12:00:18.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:18.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:18.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:00:18.952 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:00:18.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:18.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:19.106 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 110 up_thru 110 down_at 107 last_clean_interval [99,106) [v2:127.0.0.1:6810/2102600160,v1:127.0.0.1:6811/2102600160] [v2:127.0.0.1:6812/2102600160,v1:127.0.0.1:6813/2102600160] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T12:00:19.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:00:19.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:00:19.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:00:19.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:00:19.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:00:19.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:00:19.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:00:19.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:00:19.108 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:00:19.108 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:00:19.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:00:19.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:00:19.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:00:19.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:00:19.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:00:19.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:00:19.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:00:19.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:00:19.225 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:00:19.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:00:19.381 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:00:19.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:00:19.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:00:19.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:00:19.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=450971566085 2026-02-17T12:00:19.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 450971566085 2026-02-17T12:00:19.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566085' 2026-02-17T12:00:19.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:00:19.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:00:19.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=472446402562 2026-02-17T12:00:19.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 472446402562 2026-02-17T12:00:19.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566085 1-472446402562' 2026-02-17T12:00:19.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:00:19.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-450971566085 2026-02-17T12:00:19.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:00:19.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:00:19.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-450971566085 2026-02-17T12:00:19.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:00:19.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=450971566085 2026-02-17T12:00:19.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 450971566085' 2026-02-17T12:00:19.531 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 450971566085 2026-02-17T12:00:19.531 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:00:19.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566084 -lt 450971566085 2026-02-17T12:00:19.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:00:20.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:00:20.694 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:00:20.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566085 -lt 450971566085 2026-02-17T12:00:20.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:00:20.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-472446402562 2026-02-17T12:00:20.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:00:20.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:00:20.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-472446402562 2026-02-17T12:00:20.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:00:20.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=472446402562 2026-02-17T12:00:20.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 472446402562' 2026-02-17T12:00:20.856 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 472446402562 2026-02-17T12:00:20.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:00:21.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 472446402562 -lt 472446402562 2026-02-17T12:00:21.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:00:21.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:00:21.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:00:21.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:00:21.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:00:21.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:00:21.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:00:21.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:00:21.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:00:21.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:00:21.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:00:21.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:00:21.351 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:00:21.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:00:21.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:00:21.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:00:21.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:00:21.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:00:21.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1087: TEST_corrupt_scrub_replicated: echo -n val3-ROBJ8 2026-02-17T12:00:21.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1088: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ8 set-attr _key3-ROBJ8 td/osd-scrub-repair/newval 2026-02-17T12:00:21.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:00:21.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:00:21.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:00:21.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:00:21.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ8 set-attr _key3-ROBJ8 td/osd-scrub-repair/newval 2026-02-17T12:00:21.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:00:21.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:00:21.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:00:21.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:00:21.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:00:21.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:00:21.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:00:21.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:00:21.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:00:21.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:00:22.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:00:22.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ8 set-attr _key3-ROBJ8 td/osd-scrub-repair/newval 2026-02-17T12:00:22.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:00:22.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:00:22.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:00:22.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:00:22.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:00:22.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ8 set-attr _key3-ROBJ8 td/osd-scrub-repair/newval 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:00:24.022 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:00:24.023 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:00:24.023 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:00:24.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:00:24.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:00:24.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:00:24.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:00:24.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:00:24.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:00:24.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:00:24.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:00:24.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:00:24.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:00:24.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:00:24.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:00:24.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:00:24.025 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:00:24.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:00:24.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:00:24.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:00:24.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:00:24.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:00:24.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:00:24.038 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:24.033+0000 7f37973538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:24.040 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:24.033+0000 7f37973538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:24.041 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:24.033+0000 7f37973538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:24.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:00:24.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:00:24.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:00:24.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:00:24.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:00:24.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:24.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:00:24.177 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:00:24.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:24.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:24.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:25.272 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:25.265+0000 7f37973538c0 -1 Falling back to public interface 2026-02-17T12:00:25.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:25.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:25.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:00:25.337 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:00:25.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:25.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:25.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:26.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:26.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:26.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:00:26.491 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:00:26.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:26.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:26.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:27.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:27.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:27.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:00:27.648 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:00:27.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:27.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:27.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:28.285 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:28.277+0000 7f37973538c0 -1 osd.1 112 log_to_monitors true 2026-02-17T12:00:28.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:28.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:28.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:00:28.804 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:00:28.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:28.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:28.959 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 116 up_thru 116 down_at 113 last_clean_interval [110,112) [v2:127.0.0.1:6810/2086835658,v1:127.0.0.1:6811/2086835658] [v2:127.0.0.1:6812/2086835658,v1:127.0.0.1:6813/2086835658] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T12:00:28.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:00:28.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:00:28.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:00:28.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:00:28.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:00:28.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:00:28.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:00:28.959 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:00:28.960 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:00:28.960 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:00:28.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:00:28.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:00:28.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:00:29.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:00:29.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:00:29.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:00:29.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:00:29.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:00:29.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:00:29.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:00:29.226 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:00:29.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:00:29.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:00:29.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:00:29.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=450971566088 2026-02-17T12:00:29.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 450971566088 2026-02-17T12:00:29.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566088' 2026-02-17T12:00:29.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:00:29.295 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:00:29.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=498216206338 2026-02-17T12:00:29.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 498216206338 2026-02-17T12:00:29.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566088 1-498216206338' 2026-02-17T12:00:29.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:00:29.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-450971566088 2026-02-17T12:00:29.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:00:29.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:00:29.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-450971566088 2026-02-17T12:00:29.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:00:29.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=450971566088 2026-02-17T12:00:29.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 450971566088' 2026-02-17T12:00:29.373 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 450971566088 2026-02-17T12:00:29.373 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:00:29.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566087 -lt 450971566088 2026-02-17T12:00:29.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:00:30.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:00:30.535 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:00:30.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566088 -lt 450971566088 2026-02-17T12:00:30.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:00:30.693 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-498216206338 2026-02-17T12:00:30.693 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:00:30.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:00:30.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-498216206338 2026-02-17T12:00:30.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:00:30.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=498216206338 2026-02-17T12:00:30.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 498216206338' 2026-02-17T12:00:30.697 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 498216206338 2026-02-17T12:00:30.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:00:30.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 498216206338 -lt 498216206338 2026-02-17T12:00:30.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:00:30.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:00:30.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:00:31.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:00:31.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:00:31.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:00:31.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:00:31.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:00:31.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:00:31.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:00:31.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:00:31.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:00:31.189 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:00:31.189 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:00:31.189 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:00:31.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:00:31.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:00:31.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:00:31.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1089: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/bad-val td/osd-scrub-repair/newval 2026-02-17T12:00:31.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T12:00:31.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ9 2026-02-17T12:00:31.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 9 % 2 2026-02-17T12:00:31.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T12:00:31.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T12:00:31.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1093: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ9 get-attr _ 2026-02-17T12:00:31.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:00:31.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:00:31.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:00:31.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:00:31.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ9 get-attr _ 2026-02-17T12:00:31.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:00:31.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:00:31.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:00:31.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:00:31.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:00:31.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:00:31.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:00:31.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:00:31.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:00:31.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:00:32.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:00:32.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ9 get-attr _ 2026-02-17T12:00:32.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:00:32.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:00:32.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:00:32.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:00:32.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:00:32.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ9 get-attr _ 2026-02-17T12:00:33.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T12:00:33.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:00:33.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:00:33.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:00:33.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:00:33.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:00:33.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:00:33.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:00:33.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:00:33.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:00:33.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:00:33.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:00:33.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:00:33.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:00:33.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:00:33.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:00:33.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:00:33.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:00:33.552 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:00:33.552 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:00:33.552 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:00:33.552 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:00:33.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:00:33.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:00:33.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:00:33.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:00:33.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:00:33.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:00:33.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:00:33.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:00:33.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:00:33.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:00:33.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:00:33.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:00:33.553 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:00:33.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:00:33.553 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:00:33.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:00:33.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:00:33.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:00:33.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:00:33.570 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:33.561+0000 7f30c7e608c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:33.571 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:33.565+0000 7f30c7e608c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:33.572 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:33.565+0000 7f30c7e608c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:33.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:00:33.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:00:33.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:00:33.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:00:33.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:00:33.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:33.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:00:33.702 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:00:33.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:33.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:33.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:34.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:34.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:34.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:00:34.862 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:00:34.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:34.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:34.872 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:34.865+0000 7f30c7e608c0 -1 Falling back to public interface 2026-02-17T12:00:35.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:36.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:36.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:36.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:00:36.012 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:00:36.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:36.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:36.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:37.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:37.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:37.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:00:37.170 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:00:37.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:37.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:37.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:37.865 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:37.857+0000 7f30c7e608c0 -1 osd.1 117 log_to_monitors true 2026-02-17T12:00:38.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:38.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:38.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:00:38.325 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:00:38.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:38.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:38.479 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 121 up_thru 121 down_at 118 last_clean_interval [116,117) [v2:127.0.0.1:6810/2311328739,v1:127.0.0.1:6811/2311328739] [v2:127.0.0.1:6812/2311328739,v1:127.0.0.1:6813/2311328739] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T12:00:38.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:00:38.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:00:38.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:00:38.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:00:38.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:00:38.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:00:38.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:00:38.480 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:00:38.481 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:00:38.481 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:00:38.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:00:38.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:00:38.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:00:38.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:00:38.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:00:38.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:00:38.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:00:38.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:00:38.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:00:38.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:00:38.750 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:00:38.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:00:38.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:00:38.750 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:00:38.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=450971566091 2026-02-17T12:00:38.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 450971566091 2026-02-17T12:00:38.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566091' 2026-02-17T12:00:38.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:00:38.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:00:38.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=519691042818 2026-02-17T12:00:38.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 519691042818 2026-02-17T12:00:38.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566091 1-519691042818' 2026-02-17T12:00:38.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:00:38.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-450971566091 2026-02-17T12:00:38.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:00:38.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:00:38.903 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-450971566091 2026-02-17T12:00:38.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:00:38.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=450971566091 2026-02-17T12:00:38.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 450971566091' 2026-02-17T12:00:38.905 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 450971566091 2026-02-17T12:00:38.906 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:00:39.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566090 -lt 450971566091 2026-02-17T12:00:39.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:00:40.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:00:40.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:00:40.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566091 -lt 450971566091 2026-02-17T12:00:40.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:00:40.220 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-519691042818 2026-02-17T12:00:40.221 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:00:40.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:00:40.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-519691042818 2026-02-17T12:00:40.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:00:40.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=519691042818 2026-02-17T12:00:40.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 519691042818' 2026-02-17T12:00:40.225 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 519691042818 2026-02-17T12:00:40.225 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:00:40.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 519691042818 -lt 519691042818 2026-02-17T12:00:40.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:00:40.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:00:40.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:00:40.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:00:40.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:00:40.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:00:40.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:00:40.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:00:40.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:00:40.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:00:40.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:00:40.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:00:40.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:00:40.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:00:40.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:00:40.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:00:40.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:00:40.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:00:40.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1094: TEST_corrupt_scrub_replicated: echo -n D 2026-02-17T12:00:40.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1095: TEST_corrupt_scrub_replicated: rados --pool csr_pool put ROBJ9 td/osd-scrub-repair/change 2026-02-17T12:00:40.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1096: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-17T12:00:40.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:00:40.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:00:40.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:00:40.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:00:40.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-17T12:00:40.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:00:40.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:00:40.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:00:40.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:00:40.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:00:40.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:00:40.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:00:40.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:00:40.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:00:40.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:00:42.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:00:42.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-17T12:00:42.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:00:42.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:00:42.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:00:42.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:00:42.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:00:42.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-17T12:00:43.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T12:00:43.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:00:43.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:00:43.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:00:43.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:00:43.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:00:43.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:00:43.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:00:43.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:00:43.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:00:43.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:00:43.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:00:43.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:00:43.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:00:43.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:00:43.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:00:43.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:00:43.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:00:43.392 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:00:43.393 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:00:43.393 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:00:43.393 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:00:43.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:00:43.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:00:43.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:00:43.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:00:43.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:00:43.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:00:43.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:00:43.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:00:43.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:00:43.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:00:43.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:00:43.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:00:43.394 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:00:43.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:00:43.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:00:43.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:00:43.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:00:43.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:00:43.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:00:43.410 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:43.401+0000 7fd71390a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:43.411 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:43.405+0000 7fd71390a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:43.412 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:43.405+0000 7fd71390a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:43.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:00:43.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:00:43.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:00:43.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:00:43.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:00:43.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:43.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:00:43.549 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:00:43.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:43.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:43.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:44.660 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:44.653+0000 7fd71390a8c0 -1 Falling back to public interface 2026-02-17T12:00:44.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:44.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:44.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:00:44.701 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:00:44.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:44.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:44.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:45.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:45.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:45.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:00:45.851 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:00:45.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:45.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:46.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:47.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:47.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:47.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:00:47.003 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:00:47.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:47.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:47.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:47.627 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:47.621+0000 7fd71390a8c0 -1 osd.1 123 log_to_monitors true 2026-02-17T12:00:48.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:48.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:48.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:00:48.151 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:00:48.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:48.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:00:48.298 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 127 up_thru 127 down_at 124 last_clean_interval [121,123) [v2:127.0.0.1:6810/3884013562,v1:127.0.0.1:6811/3884013562] [v2:127.0.0.1:6812/3884013562,v1:127.0.0.1:6813/3884013562] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T12:00:48.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:00:48.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:00:48.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:00:48.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:00:48.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:00:48.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:00:48.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:00:48.299 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:00:48.300 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:00:48.300 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:00:48.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:00:48.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:00:48.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:00:48.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:00:48.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:00:48.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:00:48.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:00:48.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:00:48.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:00:48.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:00:48.564 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:00:48.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:00:48.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:00:48.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:00:48.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=450971566094 2026-02-17T12:00:48.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 450971566094 2026-02-17T12:00:48.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566094' 2026-02-17T12:00:48.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:00:48.636 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:00:48.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=545460846594 2026-02-17T12:00:48.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 545460846594 2026-02-17T12:00:48.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-450971566094 1-545460846594' 2026-02-17T12:00:48.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:00:48.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-450971566094 2026-02-17T12:00:48.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:00:48.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:00:48.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-450971566094 2026-02-17T12:00:48.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:00:48.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=450971566094 2026-02-17T12:00:48.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 450971566094' 2026-02-17T12:00:48.711 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 450971566094 2026-02-17T12:00:48.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:00:48.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566093 -lt 450971566094 2026-02-17T12:00:48.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:00:49.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:00:49.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:00:50.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 450971566094 -lt 450971566094 2026-02-17T12:00:50.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:00:50.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-545460846594 2026-02-17T12:00:50.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:00:50.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:00:50.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-545460846594 2026-02-17T12:00:50.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:00:50.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=545460846594 2026-02-17T12:00:50.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 545460846594' 2026-02-17T12:00:50.038 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 545460846594 2026-02-17T12:00:50.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:00:50.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 545460846594 -lt 545460846594 2026-02-17T12:00:50.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:00:50.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:00:50.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:00:50.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:00:50.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:00:50.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:00:50.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:00:50.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:00:50.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:00:50.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:00:50.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:00:50.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:00:50.538 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:00:50.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:00:50.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:00:50.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:00:50.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:00:50.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:00:50.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1097: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/oi td/osd-scrub-repair/change 2026-02-17T12:00:50.726 INFO:tasks.workunit.client.0.trial128.stderr:rm: cannot remove 'td/osd-scrub-repair/oi': No such file or directory 2026-02-17T12:00:50.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T12:00:50.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ10 2026-02-17T12:00:50.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 10 % 2 2026-02-17T12:00:50.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T12:00:50.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T12:00:50.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T12:00:50.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ11 2026-02-17T12:00:50.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 11 % 2 2026-02-17T12:00:50.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T12:00:50.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T12:00:50.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T12:00:50.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ12 2026-02-17T12:00:50.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 12 % 2 2026-02-17T12:00:50.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T12:00:50.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T12:00:50.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T12:00:50.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ13 2026-02-17T12:00:50.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 13 % 2 2026-02-17T12:00:50.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T12:00:50.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T12:00:50.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T12:00:50.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ14 2026-02-17T12:00:50.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 14 % 2 2026-02-17T12:00:50.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T12:00:50.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T12:00:50.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1106: TEST_corrupt_scrub_replicated: echo -n bad-val 2026-02-17T12:00:50.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1107: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ14 set-attr _ td/osd-scrub-repair/bad-val 2026-02-17T12:00:50.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:00:50.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:00:50.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T12:00:50.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:00:50.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ14 set-attr _ td/osd-scrub-repair/bad-val 2026-02-17T12:00:50.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:00:50.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:00:50.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T12:00:50.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:00:50.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T12:00:50.737 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:00:50.737 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:00:50.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:00:50.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:00:50.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:00:52.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:00:52.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ14 set-attr _ td/osd-scrub-repair/bad-val 2026-02-17T12:00:52.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:00:52.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:00:52.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:00:52.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:00:52.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:00:52.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ14 set-attr _ td/osd-scrub-repair/bad-val 2026-02-17T12:00:53.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T12:00:53.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:00:53.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:00:53.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:00:53.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:00:53.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:00:53.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:00:53.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:00:53.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:00:53.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:00:53.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:00:53.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:00:53.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:00:53.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:00:53.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:00:53.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:00:53.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:00:53.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:00:53.224 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:00:53.224 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:00:53.224 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:00:53.224 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:00:53.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:00:53.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:00:53.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:00:53.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:00:53.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:00:53.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:00:53.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:00:53.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:00:53.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:00:53.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:00:53.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:00:53.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:00:53.227 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T12:00:53.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:00:53.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T12:00:53.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:00:53.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:00:53.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:00:53.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:00:53.246 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:53.241+0000 7fd1c63d28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:53.248 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:53.241+0000 7fd1c63d28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:53.249 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:53.241+0000 7fd1c63d28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:00:53.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:00:53.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:00:53.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:00:53.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:00:53.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:00:53.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:53.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:00:53.380 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:00:53.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:53.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:00:53.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:54.528 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:54.521+0000 7fd1c63d28c0 -1 Falling back to public interface 2026-02-17T12:00:54.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:54.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:54.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:00:54.534 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:00:54.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:54.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:00:54.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:55.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:55.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:55.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:00:55.686 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:00:55.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:55.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:00:55.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:56.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:56.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:56.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:00:56.836 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:00:56.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:56.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:00:56.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:00:57.514 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:00:57.505+0000 7fd1c63d28c0 -1 osd.0 129 log_to_monitors true 2026-02-17T12:00:57.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:00:57.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:00:57.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:00:57.994 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:00:57.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:00:57.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:00:58.143 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 133 up_thru 133 down_at 130 last_clean_interval [105,129) [v2:127.0.0.1:6802/4269265380,v1:127.0.0.1:6803/4269265380] [v2:127.0.0.1:6804/4269265380,v1:127.0.0.1:6805/4269265380] exists,up 298610f2-1d9b-4f6c-b9da-00d205e834ad 2026-02-17T12:00:58.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:00:58.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:00:58.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:00:58.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:00:58.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:00:58.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:00:58.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:00:58.145 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:00:58.145 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:00:58.145 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:00:58.145 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:00:58.145 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:00:58.145 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:00:58.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:00:58.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:00:58.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:00:58.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:00:58.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:00:58.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:00:58.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:00:58.411 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:00:58.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:00:58.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:00:58.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:00:58.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=571230650370 2026-02-17T12:00:58.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 571230650370 2026-02-17T12:00:58.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-571230650370' 2026-02-17T12:00:58.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:00:58.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:00:58.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=545460846597 2026-02-17T12:00:58.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 545460846597 2026-02-17T12:00:58.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-571230650370 1-545460846597' 2026-02-17T12:00:58.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:00:58.555 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-571230650370 2026-02-17T12:00:58.555 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:00:58.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:00:58.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-571230650370 2026-02-17T12:00:58.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:00:58.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=571230650370 2026-02-17T12:00:58.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 571230650370' 2026-02-17T12:00:58.559 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 571230650370 2026-02-17T12:00:58.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:00:58.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 571230650370 2026-02-17T12:00:58.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:00:59.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:00:59.726 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:00:59.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 571230650370 -lt 571230650370 2026-02-17T12:00:59.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:00:59.876 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-545460846597 2026-02-17T12:00:59.877 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:00:59.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:00:59.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-545460846597 2026-02-17T12:00:59.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:00:59.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=545460846597 2026-02-17T12:00:59.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 545460846597' 2026-02-17T12:00:59.881 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 545460846597 2026-02-17T12:00:59.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:01:00.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 545460846597 -lt 545460846597 2026-02-17T12:01:00.046 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:01:00.047 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:01:00.047 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:01:00.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:01:00.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:01:00.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:01:00.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:01:00.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:01:00.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:01:00.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:01:00.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:01:00.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:01:00.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:01:00.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:01:00.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:01:00.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:01:00.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:01:00.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:01:00.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1108: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ14 rm-attr _ 2026-02-17T12:01:00.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:01:00.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:01:00.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:01:00.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:01:00.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ14 rm-attr _ 2026-02-17T12:01:00.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:01:00.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:01:00.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:01:00.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:01:00.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:01:00.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:01:00.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:01:00.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:01:00.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:01:00.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:01:01.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:01:01.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ14 rm-attr _ 2026-02-17T12:01:01.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:01:01.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:01:01.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:01:01.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:01:01.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:01:01.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ14 rm-attr _ 2026-02-17T12:01:03.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T12:01:03.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:01:03.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:01:03.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:01:03.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:01:03.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:01:03.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:01:03.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:01:03.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:01:03.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:01:03.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:01:03.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:01:03.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:01:03.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:01:03.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:01:03.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:01:03.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:01:03.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:01:03.061 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:01:03.062 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:01:03.062 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:01:03.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:01:03.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:01:03.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:01:03.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:01:03.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:01:03.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:01:03.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:01:03.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:01:03.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:01:03.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:01:03.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:01:03.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:01:03.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:01:03.063 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:01:03.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:01:03.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:01:03.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:01:03.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:01:03.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:01:03.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:01:03.086 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:03.081+0000 7f3a6aa518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:03.088 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:03.081+0000 7f3a6aa518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:03.089 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:03.081+0000 7f3a6aa518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:03.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:01:03.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:01:03.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:01:03.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:01:03.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:01:03.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:03.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:01:03.216 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:01:03.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:03.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:03.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:04.319 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:04.313+0000 7f3a6aa518c0 -1 Falling back to public interface 2026-02-17T12:01:04.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:04.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:04.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:01:04.365 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:01:04.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:04.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:04.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:05.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:05.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:05.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:01:05.524 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:01:05.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:05.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:05.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:06.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:06.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:06.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:01:06.683 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:01:06.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:06.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:06.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:07.311 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:07.305+0000 7f3a6aa518c0 -1 osd.1 134 log_to_monitors true 2026-02-17T12:01:07.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:07.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:07.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:01:07.840 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:01:07.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:07.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:07.994 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 138 up_thru 138 down_at 135 last_clean_interval [127,134) [v2:127.0.0.1:6810/1735808561,v1:127.0.0.1:6811/1735808561] [v2:127.0.0.1:6812/1735808561,v1:127.0.0.1:6813/1735808561] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T12:01:07.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:01:07.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:01:07.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:01:07.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:01:07.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:01:07.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:01:07.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:01:07.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:01:07.996 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:01:07.996 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:01:07.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:01:07.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:01:07.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:01:08.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:01:08.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:01:08.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:01:08.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:01:08.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:01:08.111 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:01:08.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:01:08.264 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:01:08.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:01:08.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:01:08.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:01:08.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=571230650373 2026-02-17T12:01:08.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 571230650373 2026-02-17T12:01:08.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-571230650373' 2026-02-17T12:01:08.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:01:08.338 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:01:08.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=592705486850 2026-02-17T12:01:08.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 592705486850 2026-02-17T12:01:08.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-571230650373 1-592705486850' 2026-02-17T12:01:08.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:01:08.409 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-571230650373 2026-02-17T12:01:08.409 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:01:08.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:01:08.411 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-571230650373 2026-02-17T12:01:08.411 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:01:08.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=571230650373 2026-02-17T12:01:08.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 571230650373' 2026-02-17T12:01:08.413 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 571230650373 2026-02-17T12:01:08.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:01:08.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 571230650372 -lt 571230650373 2026-02-17T12:01:08.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:01:09.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:01:09.576 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:01:09.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 571230650372 -lt 571230650373 2026-02-17T12:01:09.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:01:10.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:01:10.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:01:10.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 571230650373 -lt 571230650373 2026-02-17T12:01:10.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:01:10.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-592705486850 2026-02-17T12:01:10.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:01:10.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:01:10.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-592705486850 2026-02-17T12:01:10.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:01:10.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=592705486850 2026-02-17T12:01:10.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 592705486850' 2026-02-17T12:01:10.893 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 592705486850 2026-02-17T12:01:10.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:01:11.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 592705486850 -lt 592705486850 2026-02-17T12:01:11.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:01:11.052 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:01:11.052 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:01:11.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:01:11.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:01:11.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:01:11.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:01:11.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:01:11.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:01:11.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:01:11.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:01:11.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:01:11.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:01:11.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:01:11.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:01:11.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:01:11.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:01:11.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:01:11.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1109: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/bad-val 2026-02-17T12:01:11.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T12:01:11.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ15 2026-02-17T12:01:11.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 15 % 2 2026-02-17T12:01:11.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T12:01:11.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T12:01:11.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1113: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ15 rm-attr _ 2026-02-17T12:01:11.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:01:11.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:01:11.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:01:11.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:01:11.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ15 rm-attr _ 2026-02-17T12:01:11.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:01:11.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:01:11.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:01:11.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:01:11.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:01:11.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:01:11.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:01:11.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:01:11.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:01:11.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:01:12.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:01:12.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ15 rm-attr _ 2026-02-17T12:01:12.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:01:12.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:01:12.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:01:12.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:01:12.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:01:12.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ15 rm-attr _ 2026-02-17T12:01:13.498 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-17T12:01:14.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T12:01:14.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:01:14.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:01:14.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:01:14.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:01:14.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:01:14.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:01:14.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:01:14.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:01:14.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:01:14.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:01:14.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:01:14.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:01:14.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:01:14.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:01:14.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:01:14.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:01:14.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:01:14.040 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:01:14.041 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:01:14.041 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:01:14.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:01:14.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:01:14.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:01:14.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:01:14.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:01:14.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:01:14.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:01:14.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:01:14.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:01:14.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:01:14.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:01:14.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:01:14.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:01:14.043 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:01:14.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:01:14.043 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:01:14.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:01:14.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:01:14.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:01:14.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:01:14.062 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:14.056+0000 7fa1bc44d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:14.064 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:14.056+0000 7fa1bc44d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:14.065 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:14.056+0000 7fa1bc44d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:14.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:01:14.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:01:14.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:01:14.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:01:14.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:01:14.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:14.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:01:14.194 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:01:14.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:14.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:14.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:15.308 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:15.300+0000 7fa1bc44d8c0 -1 Falling back to public interface 2026-02-17T12:01:15.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:15.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:15.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:01:15.350 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:01:15.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:15.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:15.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:16.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:16.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:16.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:01:16.505 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:01:16.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:16.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:16.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:17.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:17.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:17.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:01:17.661 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:01:17.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:17.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:17.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:18.288 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:18.280+0000 7fa1bc44d8c0 -1 osd.1 140 log_to_monitors true 2026-02-17T12:01:18.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:18.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:18.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:01:18.817 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:01:18.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:18.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:18.970 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 144 up_thru 144 down_at 141 last_clean_interval [138,140) [v2:127.0.0.1:6810/3730537182,v1:127.0.0.1:6811/3730537182] [v2:127.0.0.1:6812/3730537182,v1:127.0.0.1:6813/3730537182] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T12:01:18.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:01:18.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:01:18.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:01:18.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:01:18.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:01:18.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:01:18.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:01:18.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:01:18.972 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:01:18.972 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:01:18.973 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:01:18.973 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:01:18.973 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:01:19.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:01:19.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:01:19.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:01:19.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:01:19.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:01:19.089 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:01:19.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:01:19.243 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:01:19.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:01:19.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:01:19.243 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:01:19.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=571230650376 2026-02-17T12:01:19.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 571230650376 2026-02-17T12:01:19.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-571230650376' 2026-02-17T12:01:19.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:01:19.316 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:01:19.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=618475290626 2026-02-17T12:01:19.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 618475290626 2026-02-17T12:01:19.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-571230650376 1-618475290626' 2026-02-17T12:01:19.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:01:19.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-571230650376 2026-02-17T12:01:19.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:01:19.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:01:19.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-571230650376 2026-02-17T12:01:19.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:01:19.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=571230650376 2026-02-17T12:01:19.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 571230650376' 2026-02-17T12:01:19.392 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 571230650376 2026-02-17T12:01:19.393 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:01:19.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 571230650375 -lt 571230650376 2026-02-17T12:01:19.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:01:20.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:01:20.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:01:20.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 571230650376 -lt 571230650376 2026-02-17T12:01:20.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:01:20.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-618475290626 2026-02-17T12:01:20.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:01:20.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:01:20.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-618475290626 2026-02-17T12:01:20.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:01:20.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=618475290626 2026-02-17T12:01:20.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 618475290626' 2026-02-17T12:01:20.707 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 618475290626 2026-02-17T12:01:20.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:01:20.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 618475290626 -lt 618475290626 2026-02-17T12:01:20.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:01:20.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:01:20.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:01:21.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:01:21.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:01:21.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:01:21.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:01:21.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:01:21.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:01:21.052 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:01:21.052 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:01:21.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:01:21.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:01:21.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:01:21.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:01:21.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:01:21.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:01:21.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:01:21.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T12:01:21.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ16 2026-02-17T12:01:21.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 16 % 2 2026-02-17T12:01:21.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T12:01:21.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T12:01:21.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1117: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ16 rm-attr snapset 2026-02-17T12:01:21.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:01:21.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:01:21.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T12:01:21.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:01:21.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ16 rm-attr snapset 2026-02-17T12:01:21.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:01:21.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:01:21.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T12:01:21.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:01:21.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T12:01:21.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:01:21.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:01:21.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:01:21.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:01:21.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:01:22.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:01:22.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ16 rm-attr snapset 2026-02-17T12:01:22.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:01:22.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:01:22.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:01:22.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:01:22.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:01:22.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ16 rm-attr snapset 2026-02-17T12:01:23.320 INFO:tasks.workunit.client.0.trial128.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:01:23.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:01:23.857 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:01:23.857 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:01:23.857 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:01:23.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:01:23.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:01:23.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:01:23.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:01:23.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:01:23.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:01:23.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:01:23.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:01:23.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:01:23.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:01:23.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:01:23.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:01:23.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:01:23.859 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T12:01:23.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:01:23.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T12:01:23.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:01:23.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:01:23.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:01:23.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:01:23.882 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:23.876+0000 7faed1a268c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:23.884 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:23.876+0000 7faed1a268c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:23.885 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:23.876+0000 7faed1a268c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:24.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:01:24.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:01:24.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:01:24.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:01:24.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:01:24.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:24.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:01:24.012 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:01:24.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:24.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:01:24.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:25.108 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:25.100+0000 7faed1a268c0 -1 Falling back to public interface 2026-02-17T12:01:25.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:25.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:25.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:01:25.169 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:01:25.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:25.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:01:25.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:26.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:26.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:26.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:01:26.326 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:01:26.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:26.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:01:26.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:27.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:27.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:27.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:01:27.481 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:01:27.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:27.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:01:27.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:28.079 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:28.072+0000 7faed1a268c0 -1 osd.0 145 log_to_monitors true 2026-02-17T12:01:28.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:28.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:28.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:01:28.637 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:01:28.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:28.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:01:28.787 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 149 up_thru 149 down_at 146 last_clean_interval [133,145) [v2:127.0.0.1:6802/4110754851,v1:127.0.0.1:6803/4110754851] [v2:127.0.0.1:6804/4110754851,v1:127.0.0.1:6805/4110754851] exists,up 298610f2-1d9b-4f6c-b9da-00d205e834ad 2026-02-17T12:01:28.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:01:28.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:01:28.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:01:28.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:01:28.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:01:28.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:01:28.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:01:28.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:01:28.789 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:01:28.789 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:01:28.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:01:28.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:01:28.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:01:28.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:01:28.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:01:28.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:01:28.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:01:28.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:01:28.907 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:01:29.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:01:29.065 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:01:29.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:01:29.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:01:29.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:01:29.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=639950127106 2026-02-17T12:01:29.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 639950127106 2026-02-17T12:01:29.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-639950127106' 2026-02-17T12:01:29.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:01:29.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:01:29.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=618475290629 2026-02-17T12:01:29.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 618475290629 2026-02-17T12:01:29.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-639950127106 1-618475290629' 2026-02-17T12:01:29.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:01:29.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-639950127106 2026-02-17T12:01:29.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:01:29.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:01:29.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-639950127106 2026-02-17T12:01:29.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:01:29.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=639950127106 2026-02-17T12:01:29.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 639950127106' 2026-02-17T12:01:29.214 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 639950127106 2026-02-17T12:01:29.214 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:01:29.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 571230650376 -lt 639950127106 2026-02-17T12:01:29.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:01:30.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:01:30.362 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:01:30.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 639950127106 -lt 639950127106 2026-02-17T12:01:30.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:01:30.521 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-618475290629 2026-02-17T12:01:30.521 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:01:30.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:01:30.523 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-618475290629 2026-02-17T12:01:30.523 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:01:30.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=618475290629 2026-02-17T12:01:30.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 618475290629' 2026-02-17T12:01:30.525 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 618475290629 2026-02-17T12:01:30.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:01:30.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 618475290629 -lt 618475290629 2026-02-17T12:01:30.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:01:30.686 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:01:30.686 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:01:30.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:01:30.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:01:30.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:01:30.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:01:30.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:01:30.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:01:30.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:01:30.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:01:31.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:01:31.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:01:31.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:01:31.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:01:31.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:01:31.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:01:31.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:01:31.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1118: TEST_corrupt_scrub_replicated: echo -n bad-val 2026-02-17T12:01:31.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1119: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ16 set-attr snapset td/osd-scrub-repair/bad-val 2026-02-17T12:01:31.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:01:31.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:01:31.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:01:31.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:01:31.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ16 set-attr snapset td/osd-scrub-repair/bad-val 2026-02-17T12:01:31.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:01:31.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:01:31.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:01:31.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:01:31.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:01:31.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:01:31.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:01:31.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:01:31.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:01:31.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:01:32.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:01:32.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ16 set-attr snapset td/osd-scrub-repair/bad-val 2026-02-17T12:01:32.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:01:32.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:01:32.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:01:32.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:01:32.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:01:32.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ16 set-attr snapset td/osd-scrub-repair/bad-val 2026-02-17T12:01:33.147 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-17T12:01:33.147 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:01:33.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:01:33.685 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:01:33.685 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:01:33.685 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:01:33.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:01:33.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:01:33.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:01:33.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:01:33.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:01:33.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:01:33.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:01:33.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:01:33.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:01:33.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:01:33.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:01:33.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:01:33.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:01:33.687 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:01:33.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:01:33.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:01:33.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:01:33.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:01:33.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:01:33.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:01:33.710 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:33.704+0000 7f2d691228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:33.712 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:33.704+0000 7f2d691228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:33.713 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:33.704+0000 7f2d691228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:33.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:01:33.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:01:33.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:01:33.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:01:33.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:01:33.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:33.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:01:33.844 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:01:33.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:33.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:33.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:34.952 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:34.944+0000 7f2d691228c0 -1 Falling back to public interface 2026-02-17T12:01:34.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:34.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:34.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:01:34.998 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:01:34.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:34.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:35.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:36.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:36.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:36.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:01:36.148 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:01:36.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:36.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:36.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:37.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:37.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:37.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:01:37.300 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:01:37.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:37.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:37.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:38.209 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:38.200+0000 7f2d691228c0 -1 osd.1 150 log_to_monitors true 2026-02-17T12:01:38.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:38.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:38.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:01:38.453 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:01:38.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:38.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:38.600 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 154 up_thru 154 down_at 151 last_clean_interval [144,150) [v2:127.0.0.1:6810/736917882,v1:127.0.0.1:6811/736917882] [v2:127.0.0.1:6812/736917882,v1:127.0.0.1:6813/736917882] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T12:01:38.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:01:38.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:01:38.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:01:38.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:01:38.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:01:38.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:01:38.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:01:38.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:01:38.601 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:01:38.601 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:01:38.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:01:38.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:01:38.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:01:38.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:01:38.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:01:38.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:01:38.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:01:38.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:01:38.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:01:38.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:01:38.867 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:01:38.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:01:38.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:01:38.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:01:38.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=639950127109 2026-02-17T12:01:38.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 639950127109 2026-02-17T12:01:38.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-639950127109' 2026-02-17T12:01:38.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:01:38.942 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:01:39.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=661424963586 2026-02-17T12:01:39.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 661424963586 2026-02-17T12:01:39.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-639950127109 1-661424963586' 2026-02-17T12:01:39.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:01:39.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-639950127109 2026-02-17T12:01:39.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:01:39.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:01:39.018 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-639950127109 2026-02-17T12:01:39.018 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:01:39.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=639950127109 2026-02-17T12:01:39.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 639950127109' 2026-02-17T12:01:39.020 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 639950127109 2026-02-17T12:01:39.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:01:39.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 639950127107 -lt 639950127109 2026-02-17T12:01:39.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:01:40.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:01:40.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:01:40.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 639950127109 -lt 639950127109 2026-02-17T12:01:40.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:01:40.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-661424963586 2026-02-17T12:01:40.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:01:40.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:01:40.342 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-661424963586 2026-02-17T12:01:40.342 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:01:40.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=661424963586 2026-02-17T12:01:40.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 661424963586' 2026-02-17T12:01:40.344 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 661424963586 2026-02-17T12:01:40.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:01:40.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 661424963586 -lt 661424963586 2026-02-17T12:01:40.502 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:01:40.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:01:40.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:01:40.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:01:40.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:01:40.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:01:40.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:01:40.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:01:40.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:01:40.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:01:40.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:01:40.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:01:40.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:01:40.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:01:40.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:01:41.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:01:41.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:01:41.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:01:41.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T12:01:41.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ17 2026-02-17T12:01:41.008 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 17 % 2 2026-02-17T12:01:41.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T12:01:41.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T12:01:41.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1124: TEST_corrupt_scrub_replicated: local payload=ROBJ17 2026-02-17T12:01:41.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1125: TEST_corrupt_scrub_replicated: echo ROBJ17 2026-02-17T12:01:41.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1126: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-17T12:01:41.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:01:41.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:01:41.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T12:01:41.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:01:41.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-17T12:01:41.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:01:41.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:01:41.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T12:01:41.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:01:41.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T12:01:41.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:01:41.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:01:41.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:01:41.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:01:41.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:01:42.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:01:42.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-17T12:01:42.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:01:42.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:01:42.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:01:42.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:01:42.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:01:42.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-17T12:01:42.950 INFO:tasks.workunit.client.0.trial128.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-17T12:01:43.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:01:43.488 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:01:43.489 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:01:43.489 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:01:43.489 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:01:43.489 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:01:43.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:01:43.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:01:43.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:01:43.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:01:43.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:01:43.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:01:43.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:01:43.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:01:43.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:01:43.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:01:43.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:01:43.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:01:43.491 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T12:01:43.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:01:43.492 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T12:01:43.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:01:43.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:01:43.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:01:43.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:01:43.514 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:43.508+0000 7f35d07d88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:43.515 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:43.508+0000 7f35d07d88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:43.516 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:43.508+0000 7f35d07d88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:43.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:01:43.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:01:43.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:01:43.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:01:43.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:01:43.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:43.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:01:43.642 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:01:43.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:43.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:01:43.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:44.751 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:44.744+0000 7f35d07d88c0 -1 Falling back to public interface 2026-02-17T12:01:44.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:44.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:44.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:01:44.797 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:01:44.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:44.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:01:44.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:45.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:45.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:45.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:01:45.953 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:01:45.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:45.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:01:46.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:47.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:47.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:47.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:01:47.108 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:01:47.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:47.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:01:47.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:47.711 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:47.704+0000 7f35d07d88c0 -1 osd.0 156 log_to_monitors true 2026-02-17T12:01:48.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:48.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:48.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:01:48.264 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:01:48.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:48.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:01:48.415 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 160 up_thru 160 down_at 157 last_clean_interval [149,156) [v2:127.0.0.1:6802/486757152,v1:127.0.0.1:6803/486757152] [v2:127.0.0.1:6804/486757152,v1:127.0.0.1:6805/486757152] exists,up 298610f2-1d9b-4f6c-b9da-00d205e834ad 2026-02-17T12:01:48.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:01:48.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:01:48.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:01:48.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:01:48.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:01:48.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:01:48.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:01:48.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:01:48.416 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:01:48.416 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:01:48.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:01:48.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:01:48.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:01:48.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:01:48.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:01:48.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:01:48.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:01:48.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:01:48.534 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:01:48.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:01:48.689 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:01:48.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:01:48.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:01:48.690 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:01:48.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=687194767362 2026-02-17T12:01:48.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 687194767362 2026-02-17T12:01:48.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-687194767362' 2026-02-17T12:01:48.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:01:48.762 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:01:49.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=661424963589 2026-02-17T12:01:49.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 661424963589 2026-02-17T12:01:49.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-687194767362 1-661424963589' 2026-02-17T12:01:49.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:01:49.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-687194767362 2026-02-17T12:01:49.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:01:49.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:01:49.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-687194767362 2026-02-17T12:01:49.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:01:49.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=687194767362 2026-02-17T12:01:49.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 687194767362' 2026-02-17T12:01:49.006 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 687194767362 2026-02-17T12:01:49.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:01:49.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 687194767362 2026-02-17T12:01:49.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:01:49.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:01:49.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:01:50.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 687194767362 -lt 687194767362 2026-02-17T12:01:50.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:01:50.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-661424963589 2026-02-17T12:01:50.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:01:50.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:01:50.151 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-661424963589 2026-02-17T12:01:50.151 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:01:50.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=661424963589 2026-02-17T12:01:50.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 661424963589' 2026-02-17T12:01:50.153 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 661424963589 2026-02-17T12:01:50.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:01:50.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 661424963589 -lt 661424963589 2026-02-17T12:01:50.316 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:01:50.316 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:01:50.316 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:01:50.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:01:50.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:01:50.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:01:50.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:01:50.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:01:50.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:01:50.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:01:50.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:01:50.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:01:50.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:01:50.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:01:50.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:01:50.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:01:50.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:01:50.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:01:50.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1127: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-17T12:01:50.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:01:50.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:01:50.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:01:50.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:01:50.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-17T12:01:50.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:01:50.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:01:50.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:01:50.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:01:50.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:01:50.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:01:50.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:01:50.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:01:50.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:01:50.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:01:52.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:01:52.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-17T12:01:52.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:01:52.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:01:52.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:01:52.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:01:52.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:01:52.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ17 set-bytes td/osd-scrub-repair/new.ROBJ17 2026-02-17T12:01:52.785 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-17T12:01:52.785 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-17T12:01:53.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T12:01:53.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:01:53.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:01:53.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:01:53.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:01:53.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:01:53.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:01:53.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:01:53.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:01:53.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:01:53.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:01:53.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:01:53.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:01:53.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:01:53.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:01:53.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:01:53.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:01:53.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:01:53.327 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:01:53.327 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:01:53.328 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:01:53.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:01:53.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:01:53.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:01:53.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:01:53.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:01:53.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:01:53.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:01:53.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:01:53.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:01:53.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:01:53.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:01:53.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:01:53.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:01:53.331 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:01:53.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:01:53.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:01:53.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:01:53.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:01:53.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:01:53.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:01:53.349 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:53.340+0000 7fcd48ce18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:53.351 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:53.344+0000 7fcd48ce18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:53.352 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:53.344+0000 7fcd48ce18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:01:53.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:01:53.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:01:53.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:01:53.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:01:53.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:01:53.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:53.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:01:53.481 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:01:53.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:53.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:53.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:54.579 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:54.572+0000 7fcd48ce18c0 -1 Falling back to public interface 2026-02-17T12:01:54.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:54.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:54.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:01:54.639 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:01:54.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:54.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:54.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:55.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:55.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:55.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:01:55.796 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:01:55.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:55.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:55.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:56.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:56.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:56.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:01:56.951 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:01:56.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:56.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:57.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:01:57.542 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:01:57.532+0000 7fcd48ce18c0 -1 osd.1 161 log_to_monitors true 2026-02-17T12:01:58.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:01:58.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:01:58.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:01:58.106 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:01:58.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:01:58.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:01:58.257 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 165 up_thru 165 down_at 162 last_clean_interval [154,161) [v2:127.0.0.1:6810/1390268629,v1:127.0.0.1:6811/1390268629] [v2:127.0.0.1:6812/1390268629,v1:127.0.0.1:6813/1390268629] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T12:01:58.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:01:58.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:01:58.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:01:58.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:01:58.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:01:58.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:01:58.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:01:58.258 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:01:58.258 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:01:58.258 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:01:58.258 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:01:58.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:01:58.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:01:58.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:01:58.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:01:58.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:01:58.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:01:58.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:01:58.377 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:01:58.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:01:58.523 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:01:58.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:01:58.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:01:58.523 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:01:58.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=687194767365 2026-02-17T12:01:58.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 687194767365 2026-02-17T12:01:58.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-687194767365' 2026-02-17T12:01:58.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:01:58.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:01:58.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=708669603842 2026-02-17T12:01:58.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 708669603842 2026-02-17T12:01:58.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-687194767365 1-708669603842' 2026-02-17T12:01:58.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:01:58.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-687194767365 2026-02-17T12:01:58.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:01:58.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:01:58.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-687194767365 2026-02-17T12:01:58.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:01:58.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=687194767365 2026-02-17T12:01:58.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 687194767365' 2026-02-17T12:01:58.672 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 687194767365 2026-02-17T12:01:58.672 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:01:58.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 687194767364 -lt 687194767365 2026-02-17T12:01:58.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:01:59.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:01:59.840 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:01:59.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 687194767365 -lt 687194767365 2026-02-17T12:01:59.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:01:59.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-708669603842 2026-02-17T12:01:59.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:01:59.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:01:59.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-708669603842 2026-02-17T12:01:59.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:01:59.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=708669603842 2026-02-17T12:01:59.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 708669603842' 2026-02-17T12:01:59.996 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 708669603842 2026-02-17T12:01:59.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:02:00.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 708669603842 -lt 708669603842 2026-02-17T12:02:00.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:02:00.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:02:00.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:02:00.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:02:00.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:02:00.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:02:00.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:02:00.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:02:00.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:02:00.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:02:00.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:02:00.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:02:00.502 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:02:00.502 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:02:00.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:02:00.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:02:00.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:02:00.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:02:00.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T12:02:00.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ18 2026-02-17T12:02:00.690 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 18 % 2 2026-02-17T12:02:00.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=0 2026-02-17T12:02:00.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T12:02:00.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1132: TEST_corrupt_scrub_replicated: local payload=ROBJ18 2026-02-17T12:02:00.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1133: TEST_corrupt_scrub_replicated: echo ROBJ18 2026-02-17T12:02:00.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1134: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-17T12:02:00.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:02:00.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:02:00.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T12:02:00.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:02:00.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-17T12:02:00.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:02:00.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:02:00.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T12:02:00.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:02:00.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T12:02:00.694 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:02:00.694 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:02:00.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:02:00.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:02:00.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:02:02.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:02:02.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-17T12:02:02.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:02:02.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:02:02.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:02:02.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:02:02.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:02:02.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-17T12:02:02.648 INFO:tasks.workunit.client.0.trial128.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-17T12:02:03.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T12:02:03.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:02:03.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:02:03.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:02:03.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:02:03.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:02:03.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:02:03.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:02:03.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:02:03.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:02:03.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:02:03.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:02:03.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:02:03.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:02:03.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:02:03.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:02:03.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:03.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:03.187 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:03.187 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:03.187 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:02:03.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:02:03.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:02:03.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:02:03.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:02:03.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:02:03.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:02:03.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:02:03.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:02:03.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:02:03.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:02:03.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:02:03.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:02:03.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:02:03.191 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T12:02:03.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:02:03.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T12:02:03.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:02:03.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:02:03.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:02:03.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:02:03.210 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:03.204+0000 7f1a9b8a08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:03.211 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:03.204+0000 7f1a9b8a08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:03.212 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:03.204+0000 7f1a9b8a08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:03.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:02:03.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:02:03.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:02:03.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:02:03.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:02:03.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:03.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:02:03.346 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:02:03.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:03.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:03.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:04.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:04.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:04.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:02:04.500 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:02:04.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:04.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:04.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:04.667 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:04.660+0000 7f1a9b8a08c0 -1 Falling back to public interface 2026-02-17T12:02:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:02:05.653 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:02:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:05.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:05.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:06.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:06.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:06.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:02:06.805 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:02:06.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:06.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:06.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:07.646 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:07.640+0000 7f1a9b8a08c0 -1 osd.0 166 log_to_monitors true 2026-02-17T12:02:07.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:07.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:07.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:02:07.956 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:02:07.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:07.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:08.105 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 170 up_thru 170 down_at 167 last_clean_interval [160,166) [v2:127.0.0.1:6802/400941620,v1:127.0.0.1:6803/400941620] [v2:127.0.0.1:6804/400941620,v1:127.0.0.1:6805/400941620] exists,up 298610f2-1d9b-4f6c-b9da-00d205e834ad 2026-02-17T12:02:08.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:02:08.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:02:08.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:02:08.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:02:08.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:02:08.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:02:08.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:02:08.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:02:08.107 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:02:08.107 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:02:08.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:02:08.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:02:08.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:02:08.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:02:08.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:02:08.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:02:08.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:02:08.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:02:08.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:02:08.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:02:08.368 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:02:08.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:02:08.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:02:08.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:02:08.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=730144440322 2026-02-17T12:02:08.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 730144440322 2026-02-17T12:02:08.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-730144440322' 2026-02-17T12:02:08.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:02:08.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:02:08.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=708669603845 2026-02-17T12:02:08.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 708669603845 2026-02-17T12:02:08.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-730144440322 1-708669603845' 2026-02-17T12:02:08.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:02:08.512 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-730144440322 2026-02-17T12:02:08.512 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:02:08.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:02:08.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-730144440322 2026-02-17T12:02:08.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:02:08.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=730144440322 2026-02-17T12:02:08.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 730144440322' 2026-02-17T12:02:08.516 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 730144440322 2026-02-17T12:02:08.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:02:08.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 730144440322 2026-02-17T12:02:08.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:02:09.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:02:09.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:02:09.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 730144440322 2026-02-17T12:02:09.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:02:10.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:02:10.832 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:02:10.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 730144440322 -lt 730144440322 2026-02-17T12:02:10.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:02:10.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-708669603845 2026-02-17T12:02:10.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:02:10.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:02:10.986 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-708669603845 2026-02-17T12:02:10.986 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:02:10.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=708669603845 2026-02-17T12:02:10.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 708669603845' 2026-02-17T12:02:10.989 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 708669603845 2026-02-17T12:02:10.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:02:11.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 708669603845 -lt 708669603845 2026-02-17T12:02:11.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:02:11.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:02:11.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:02:11.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:02:11.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:02:11.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:02:11.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:02:11.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:02:11.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:02:11.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:02:11.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:02:11.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:02:11.492 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:02:11.492 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:02:11.492 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:02:11.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:02:11.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:02:11.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:02:11.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1135: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-17T12:02:11.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:02:11.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:02:11.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:02:11.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:02:11.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-17T12:02:11.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:02:11.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:02:11.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:02:11.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:02:11.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:02:11.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:02:11.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:02:11.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:02:11.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:02:11.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:02:12.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:02:12.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-17T12:02:12.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:02:12.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:02:12.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:02:12.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:02:12.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:02:12.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ18 set-bytes td/osd-scrub-repair/new.ROBJ18 2026-02-17T12:02:13.637 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-17T12:02:13.637 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-17T12:02:14.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T12:02:14.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:02:14.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:02:14.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:02:14.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:02:14.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:02:14.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:02:14.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:02:14.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:02:14.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:02:14.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:02:14.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:02:14.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:02:14.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:02:14.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:02:14.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:02:14.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:14.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:14.172 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:14.172 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:14.172 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:02:14.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:02:14.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:02:14.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:02:14.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:02:14.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:02:14.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:02:14.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:02:14.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:02:14.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:02:14.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:02:14.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:02:14.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:02:14.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:02:14.174 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:02:14.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:02:14.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:02:14.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:02:14.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:02:14.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:02:14.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:02:14.193 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:14.184+0000 7f740cbf58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:14.194 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:14.188+0000 7f740cbf58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:14.195 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:14.188+0000 7f740cbf58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:14.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:02:14.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:02:14.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:02:14.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:02:14.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:02:14.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:14.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:02:14.333 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:02:14.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:14.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:14.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:15.467 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:15.460+0000 7f740cbf58c0 -1 Falling back to public interface 2026-02-17T12:02:15.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:15.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:15.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:02:15.484 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:02:15.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:15.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:15.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:16.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:16.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:16.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:02:16.634 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:02:16.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:16.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:16.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:17.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:17.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:17.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:02:17.789 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:02:17.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:17.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:17.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:18.488 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:18.480+0000 7f740cbf58c0 -1 osd.1 172 log_to_monitors true 2026-02-17T12:02:18.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:18.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:18.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:02:18.941 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:02:18.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:18.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:19.093 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 176 up_thru 176 down_at 173 last_clean_interval [165,172) [v2:127.0.0.1:6810/2601660326,v1:127.0.0.1:6811/2601660326] [v2:127.0.0.1:6812/2601660326,v1:127.0.0.1:6813/2601660326] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T12:02:19.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:02:19.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:02:19.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:02:19.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:02:19.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:02:19.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:02:19.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:02:19.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:02:19.094 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:02:19.095 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:02:19.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:02:19.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:02:19.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:02:19.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:02:19.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:02:19.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:02:19.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:02:19.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:02:19.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:02:19.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:02:19.362 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:02:19.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:02:19.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:02:19.362 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:02:19.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=730144440325 2026-02-17T12:02:19.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 730144440325 2026-02-17T12:02:19.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-730144440325' 2026-02-17T12:02:19.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:02:19.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:02:19.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=755914244098 2026-02-17T12:02:19.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 755914244098 2026-02-17T12:02:19.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-730144440325 1-755914244098' 2026-02-17T12:02:19.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:02:19.509 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-730144440325 2026-02-17T12:02:19.509 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:02:19.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:02:19.511 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-730144440325 2026-02-17T12:02:19.511 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:02:19.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=730144440325 2026-02-17T12:02:19.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 730144440325' 2026-02-17T12:02:19.513 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 730144440325 2026-02-17T12:02:19.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:02:19.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 730144440324 -lt 730144440325 2026-02-17T12:02:19.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:02:20.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:02:20.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:02:20.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 730144440325 -lt 730144440325 2026-02-17T12:02:20.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:02:20.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-755914244098 2026-02-17T12:02:20.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:02:20.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:02:20.837 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-755914244098 2026-02-17T12:02:20.837 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:02:20.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=755914244098 2026-02-17T12:02:20.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 755914244098' 2026-02-17T12:02:20.839 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 755914244098 2026-02-17T12:02:20.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:02:20.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 755914244098 -lt 755914244098 2026-02-17T12:02:20.990 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:02:20.991 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:02:20.991 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:02:21.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:02:21.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:02:21.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:02:21.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:02:21.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:02:21.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:02:21.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:02:21.183 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:02:21.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:02:21.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:02:21.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:02:21.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:02:21.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:02:21.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:02:21.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:02:21.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1137: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ18 corrupt-info 2026-02-17T12:02:21.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:02:21.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:02:21.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:02:21.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:02:21.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ18 corrupt-info 2026-02-17T12:02:21.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:02:21.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:02:21.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:02:21.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:02:21.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:02:21.527 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:02:21.527 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:02:21.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:02:21.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:02:21.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:02:22.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:02:22.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ18 corrupt-info 2026-02-17T12:02:22.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:02:22.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:02:22.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:02:22.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:02:22.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:02:22.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ18 corrupt-info 2026-02-17T12:02:23.475 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-17T12:02:23.475 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-17T12:02:24.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T12:02:24.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:02:24.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:02:24.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:02:24.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:02:24.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:02:24.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:02:24.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:02:24.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:02:24.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:02:24.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:02:24.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:02:24.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:02:24.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:02:24.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:02:24.014 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:02:24.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:24.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:24.015 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:24.015 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:24.015 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:02:24.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:02:24.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:02:24.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:02:24.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:02:24.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:02:24.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:02:24.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:02:24.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:02:24.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:02:24.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:02:24.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:02:24.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:02:24.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:02:24.018 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:02:24.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:02:24.018 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:02:24.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:02:24.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:02:24.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:02:24.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:02:24.038 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:24.032+0000 7f51e856a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:24.039 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:24.032+0000 7f51e856a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:24.040 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:24.032+0000 7f51e856a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:24.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:02:24.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:02:24.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:02:24.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:02:24.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:02:24.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:24.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:02:24.169 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:02:24.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:24.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:24.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:25.315 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:25.308+0000 7f51e856a8c0 -1 Falling back to public interface 2026-02-17T12:02:25.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:25.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:25.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:02:25.330 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:02:25.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:25.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:25.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:26.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:26.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:26.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:02:26.483 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:02:26.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:26.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:26.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:27.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:27.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:27.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:02:27.636 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:02:27.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:27.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:27.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:28.274 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:28.267+0000 7f51e856a8c0 -1 osd.1 177 log_to_monitors true 2026-02-17T12:02:28.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:28.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:28.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:02:28.791 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:02:28.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:28.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:28.947 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 181 up_thru 181 down_at 178 last_clean_interval [176,177) [v2:127.0.0.1:6810/1915642010,v1:127.0.0.1:6811/1915642010] [v2:127.0.0.1:6812/1915642010,v1:127.0.0.1:6813/1915642010] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T12:02:28.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:02:28.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:02:28.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:02:28.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:02:28.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:02:28.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:02:28.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:02:28.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:02:28.949 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:02:28.949 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:02:28.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:02:28.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:02:28.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:02:29.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:02:29.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:02:29.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:02:29.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:02:29.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:02:29.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:02:29.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:02:29.220 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:02:29.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:02:29.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:02:29.221 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:02:29.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=730144440328 2026-02-17T12:02:29.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 730144440328 2026-02-17T12:02:29.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-730144440328' 2026-02-17T12:02:29.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:02:29.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:02:29.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=777389080578 2026-02-17T12:02:29.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 777389080578 2026-02-17T12:02:29.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-730144440328 1-777389080578' 2026-02-17T12:02:29.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:02:29.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-730144440328 2026-02-17T12:02:29.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:02:29.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:02:29.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-730144440328 2026-02-17T12:02:29.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:02:29.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=730144440328 2026-02-17T12:02:29.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 730144440328' 2026-02-17T12:02:29.371 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 730144440328 2026-02-17T12:02:29.372 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:02:29.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 730144440327 -lt 730144440328 2026-02-17T12:02:29.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:02:30.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:02:30.535 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:02:30.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 730144440328 -lt 730144440328 2026-02-17T12:02:30.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:02:30.683 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-777389080578 2026-02-17T12:02:30.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:02:30.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:02:30.686 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-777389080578 2026-02-17T12:02:30.686 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:02:30.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=777389080578 2026-02-17T12:02:30.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 777389080578' 2026-02-17T12:02:30.688 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 777389080578 2026-02-17T12:02:30.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:02:30.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 777389080578 -lt 777389080578 2026-02-17T12:02:30.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:02:30.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:02:30.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:02:31.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:02:31.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:02:31.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:02:31.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:02:31.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:02:31.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:02:31.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:02:31.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:02:31.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:02:31.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:02:31.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:02:31.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:02:31.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:02:31.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:02:31.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:02:31.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1029: TEST_corrupt_scrub_replicated: for i in $(seq 1 $total_objs) 2026-02-17T12:02:31.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1030: TEST_corrupt_scrub_replicated: objname=ROBJ19 2026-02-17T12:02:31.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: expr 19 % 2 2026-02-17T12:02:31.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1033: TEST_corrupt_scrub_replicated: local osd=1 2026-02-17T12:02:31.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1035: TEST_corrupt_scrub_replicated: case $i in 2026-02-17T12:02:31.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1146: TEST_corrupt_scrub_replicated: get_pg csr_pool ROBJ0 2026-02-17T12:02:31.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=csr_pool 2026-02-17T12:02:31.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=ROBJ0 2026-02-17T12:02:31.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map csr_pool ROBJ0 2026-02-17T12:02:31.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T12:02:31.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1146: TEST_corrupt_scrub_replicated: local pg=3.0 2026-02-17T12:02:31.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1148: TEST_corrupt_scrub_replicated: ceph tell 'osd.*' injectargs -- --osd-max-object-size=1048576 2026-02-17T12:02:31.594 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: {} 2026-02-17T12:02:31.594 INFO:tasks.workunit.client.0.trial128.stderr:osd.0: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_max_object_size = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2026-02-17T12:02:31.602 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: {} 2026-02-17T12:02:31.603 INFO:tasks.workunit.client.0.trial128.stderr:osd.1: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_max_object_size = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2026-02-17T12:02:31.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1150: TEST_corrupt_scrub_replicated: inject_eio rep data csr_pool ROBJ11 td/osd-scrub-repair 0 2026-02-17T12:02:31.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T12:02:31.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:02:31.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:02:31.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:02:31.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=csr_pool 2026-02-17T12:02:31.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:02:31.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=ROBJ11 2026-02-17T12:02:31.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:02:31.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/osd-scrub-repair 2026-02-17T12:02:31.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:02:31.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-17T12:02:31.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:02:31.612 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds csr_pool ROBJ11 2026-02-17T12:02:31.612 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=csr_pool 2026-02-17T12:02:31.612 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=ROBJ11 2026-02-17T12:02:31.613 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map csr_pool ROBJ11 2026-02-17T12:02:31.613 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:02:31.767 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T12:02:31.767 INFO:tasks.workunit.client.0.trial128.stderr:0' 2026-02-17T12:02:31.767 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2026-02-17T12:02:31.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0') 2026-02-17T12:02:31.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:02:31.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=1 2026-02-17T12:02:31.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T12:02:31.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T12:02:31.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/osd-scrub-repair/1/type 2026-02-17T12:02:31.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:02:31.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-17T12:02:31.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:02:31.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T12:02:31.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:02:31.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:02:31.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:02:31.771 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T12:02:31.771 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:02:31.771 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:02:31.771 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:31.771 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:31.771 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:02:31.771 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.1.asok 2026-02-17T12:02:31.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40886/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:02:31.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:02:31.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:02:31.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:02:31.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.1 2026-02-17T12:02:31.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:02:31.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:02:31.842 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:31.842 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:31.842 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:02:31.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.1.asok 2026-02-17T12:02:31.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:02:31.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.40886/ceph-osd.1.asok injectdataerr csr_pool ROBJ11 2026-02-17T12:02:31.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1151: TEST_corrupt_scrub_replicated: inject_eio rep mdata csr_pool ROBJ12 td/osd-scrub-repair 1 2026-02-17T12:02:31.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T12:02:31.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:02:31.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=mdata 2026-02-17T12:02:31.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:02:31.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=csr_pool 2026-02-17T12:02:31.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:02:31.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=ROBJ12 2026-02-17T12:02:31.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:02:31.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/osd-scrub-repair 2026-02-17T12:02:31.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:02:31.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=1 2026-02-17T12:02:31.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:02:31.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds csr_pool ROBJ12 2026-02-17T12:02:31.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=csr_pool 2026-02-17T12:02:31.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=ROBJ12 2026-02-17T12:02:31.901 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map csr_pool ROBJ12 2026-02-17T12:02:31.901 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:02:32.058 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T12:02:32.058 INFO:tasks.workunit.client.0.trial128.stderr:0' 2026-02-17T12:02:32.058 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2026-02-17T12:02:32.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0') 2026-02-17T12:02:32.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:02:32.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=0 2026-02-17T12:02:32.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T12:02:32.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T12:02:32.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/osd-scrub-repair/0/type 2026-02-17T12:02:32.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:02:32.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-17T12:02:32.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:02:32.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=0 2026-02-17T12:02:32.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:02:32.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:02:32.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:02:32.061 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.0 2026-02-17T12:02:32.061 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:02:32.061 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:02:32.061 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:32.062 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:32.062 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:02:32.062 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.0.asok 2026-02-17T12:02:32.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40886/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:02:32.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:02:32.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:02:32.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:02:32.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.0 2026-02-17T12:02:32.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:02:32.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:02:32.123 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:32.123 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:32.124 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:02:32.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.0.asok 2026-02-17T12:02:32.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:02:32.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.40886/ceph-osd.0.asok injectmdataerr csr_pool ROBJ12 2026-02-17T12:02:32.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1152: TEST_corrupt_scrub_replicated: inject_eio rep mdata csr_pool ROBJ13 td/osd-scrub-repair 1 2026-02-17T12:02:32.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T12:02:32.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:02:32.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=mdata 2026-02-17T12:02:32.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:02:32.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=csr_pool 2026-02-17T12:02:32.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:02:32.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=ROBJ13 2026-02-17T12:02:32.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:02:32.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/osd-scrub-repair 2026-02-17T12:02:32.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:02:32.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=1 2026-02-17T12:02:32.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:02:32.183 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds csr_pool ROBJ13 2026-02-17T12:02:32.183 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=csr_pool 2026-02-17T12:02:32.183 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=ROBJ13 2026-02-17T12:02:32.184 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map csr_pool ROBJ13 2026-02-17T12:02:32.184 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:02:32.342 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T12:02:32.342 INFO:tasks.workunit.client.0.trial128.stderr:0' 2026-02-17T12:02:32.342 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2026-02-17T12:02:32.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0') 2026-02-17T12:02:32.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:02:32.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=0 2026-02-17T12:02:32.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T12:02:32.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T12:02:32.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/osd-scrub-repair/0/type 2026-02-17T12:02:32.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:02:32.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-17T12:02:32.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:02:32.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=0 2026-02-17T12:02:32.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:02:32.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:02:32.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:02:32.345 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.0 2026-02-17T12:02:32.346 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:02:32.346 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:02:32.346 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:32.346 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:32.346 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:02:32.346 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.0.asok 2026-02-17T12:02:32.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40886/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:02:32.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:02:32.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:02:32.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:02:32.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.0 2026-02-17T12:02:32.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:02:32.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:02:32.408 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:32.408 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:32.408 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:02:32.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.0.asok 2026-02-17T12:02:32.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:02:32.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.40886/ceph-osd.0.asok injectmdataerr csr_pool ROBJ13 2026-02-17T12:02:32.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1153: TEST_corrupt_scrub_replicated: inject_eio rep data csr_pool ROBJ13 td/osd-scrub-repair 0 2026-02-17T12:02:32.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T12:02:32.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:02:32.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:02:32.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:02:32.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=csr_pool 2026-02-17T12:02:32.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:02:32.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=ROBJ13 2026-02-17T12:02:32.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:02:32.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/osd-scrub-repair 2026-02-17T12:02:32.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:02:32.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-17T12:02:32.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:02:32.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds csr_pool ROBJ13 2026-02-17T12:02:32.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=csr_pool 2026-02-17T12:02:32.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=ROBJ13 2026-02-17T12:02:32.468 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map csr_pool ROBJ13 2026-02-17T12:02:32.468 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:02:32.629 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T12:02:32.629 INFO:tasks.workunit.client.0.trial128.stderr:0' 2026-02-17T12:02:32.629 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2026-02-17T12:02:32.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0') 2026-02-17T12:02:32.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:02:32.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=1 2026-02-17T12:02:32.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T12:02:32.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T12:02:32.629 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/osd-scrub-repair/1/type 2026-02-17T12:02:32.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:02:32.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-17T12:02:32.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:02:32.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T12:02:32.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:02:32.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:02:32.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:02:32.632 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T12:02:32.632 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:02:32.632 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:02:32.633 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:32.633 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:32.633 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:02:32.633 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.1.asok 2026-02-17T12:02:32.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40886/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:02:32.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:02:32.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:02:32.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:02:32.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.1 2026-02-17T12:02:32.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:02:32.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:02:32.696 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:02:32.696 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:32.696 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:02:32.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.1.asok 2026-02-17T12:02:32.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:02:32.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.40886/ceph-osd.1.asok injectdataerr csr_pool ROBJ13 2026-02-17T12:02:32.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1155: TEST_corrupt_scrub_replicated: pg_scrub 3.0 2026-02-17T12:02:32.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=3.0 2026-02-17T12:02:32.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 3.0 2026-02-17T12:02:32.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=3.0 2026-02-17T12:02:32.754 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T12:02:32.755 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:02:32.755 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:02:32.755 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:02:32.755 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:02:32.755 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:02:33.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T12:02:33.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T12:02:33.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T12:02:33.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T12:02:33.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:02:33.060 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:02:33.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:02:33.210 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:02:33.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:02:33.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:02:33.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:02:33.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=730144440330 2026-02-17T12:02:33.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 730144440330 2026-02-17T12:02:33.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-730144440330' 2026-02-17T12:02:33.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:02:33.287 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:02:33.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=777389080579 2026-02-17T12:02:33.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 777389080579 2026-02-17T12:02:33.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-730144440330 1-777389080579' 2026-02-17T12:02:33.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:02:33.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-730144440330 2026-02-17T12:02:33.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:02:33.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:02:33.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-730144440330 2026-02-17T12:02:33.361 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:02:33.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=730144440330 2026-02-17T12:02:33.362 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 730144440330 2026-02-17T12:02:33.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 730144440330' 2026-02-17T12:02:33.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:02:33.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 730144440328 -lt 730144440330 2026-02-17T12:02:33.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:02:34.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:02:34.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:02:34.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 730144440330 -lt 730144440330 2026-02-17T12:02:34.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:02:34.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-777389080579 2026-02-17T12:02:34.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:02:34.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:02:34.681 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-777389080579 2026-02-17T12:02:34.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:02:34.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=777389080579 2026-02-17T12:02:34.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 777389080579' 2026-02-17T12:02:34.684 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 777389080579 2026-02-17T12:02:34.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:02:34.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 777389080580 -lt 777389080579 2026-02-17T12:02:34.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T12:02:34.848 INFO:tasks.workunit.client.0.trial128.stdout:#---------- 3.0 loop 0 2026-02-17T12:02:34.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-17T12:02:34.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 3.0 2026-02-17T12:02:34.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=3.0 2026-02-17T12:02:34.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T12:02:34.849 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 3.0 query 2026-02-17T12:02:34.850 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T12:02:34.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T12:02:34.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T12:02:34.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T12:02:34.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T12:02:34.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 3.0 2026-02-17T12:02:34.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:02:34.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:02:34.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:02:34.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:02:35.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-17T11:58:47.837225+0000 2026-02-17T12:02:35.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 3.0 2026-02-17T12:02:35.215 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 3.0 on osd.1 to scrub 2026-02-17T12:02:35.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 3.0 2026-02-17T11:58:47.837225+0000 2026-02-17T12:02:35.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T12:02:35.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:58:47.837225+0000 2026-02-17T12:02:35.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:02:35.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:02:35.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:02:35.224 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T12:02:35.224 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:02:35.224 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:02:35.224 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:02:35.225 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:02:35.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:58:47.837225+0000 '>' 2026-02-17T11:58:47.837225+0000 2026-02-17T12:02:35.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:02:36.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:02:36.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:02:36.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T12:02:36.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:02:36.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:02:36.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:02:36.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:02:36.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:58:47.837225+0000 '>' 2026-02-17T11:58:47.837225+0000 2026-02-17T12:02:36.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:02:37.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:02:37.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:02:37.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T12:02:37.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:02:37.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:02:37.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:02:37.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:02:37.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:58:47.837225+0000 '>' 2026-02-17T11:58:47.837225+0000 2026-02-17T12:02:37.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:02:38.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:02:38.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:02:38.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T12:02:38.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:02:38.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:02:38.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:02:38.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:02:38.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:58:47.837225+0000 '>' 2026-02-17T11:58:47.837225+0000 2026-02-17T12:02:38.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:02:39.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:02:39.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:02:39.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T12:02:39.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:02:39.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:02:39.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:02:39.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:02:40.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:02:35.353615+0000 '>' 2026-02-17T11:58:47.837225+0000 2026-02-17T12:02:40.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:02:40.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1157: TEST_corrupt_scrub_replicated: ERRORS=0 2026-02-17T12:02:40.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1158: TEST_corrupt_scrub_replicated: declare -a s_err_strings 2026-02-17T12:02:40.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1159: TEST_corrupt_scrub_replicated: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:30259878:::ROBJ15:head : candidate had a missing info key' 2026-02-17T12:02:40.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1160: TEST_corrupt_scrub_replicated: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:33aca486:::ROBJ18:head : object info inconsistent ' 2026-02-17T12:02:40.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1161: TEST_corrupt_scrub_replicated: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:5c7b2c47:::ROBJ16:head : candidate had a corrupt snapset' 2026-02-17T12:02:40.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1162: TEST_corrupt_scrub_replicated: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:5c7b2c47:::ROBJ16:head : candidate had a missing snapset key' 2026-02-17T12:02:40.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1163: TEST_corrupt_scrub_replicated: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:5c7b2c47:::ROBJ16:head : failed to pick suitable object info' 2026-02-17T12:02:40.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1164: TEST_corrupt_scrub_replicated: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:86586531:::ROBJ8:head : attr value mismatch '\''_key1-ROBJ8'\'', attr name mismatch '\''_key3-ROBJ8'\'', attr name mismatch '\''_key2-ROBJ8'\''' 2026-02-17T12:02:40.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1165: TEST_corrupt_scrub_replicated: err_strings[6]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:bc819597:::ROBJ12:head : candidate had a stat error' 2026-02-17T12:02:40.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1166: TEST_corrupt_scrub_replicated: err_strings[7]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:c0c86b1d:::ROBJ14:head : candidate had a missing info key' 2026-02-17T12:02:40.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1167: TEST_corrupt_scrub_replicated: err_strings[8]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:c0c86b1d:::ROBJ14:head : candidate had a corrupt info' 2026-02-17T12:02:40.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1168: TEST_corrupt_scrub_replicated: err_strings[9]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:c0c86b1d:::ROBJ14:head : failed to pick suitable object info' 2026-02-17T12:02:40.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1169: TEST_corrupt_scrub_replicated: err_strings[10]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : candidate size 9 info size 7 mismatch' 2026-02-17T12:02:40.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1170: TEST_corrupt_scrub_replicated: err_strings[11]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : size 9 != size 7 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from shard 0' 2026-02-17T12:02:40.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1171: TEST_corrupt_scrub_replicated: err_strings[12]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:d60617f9:::ROBJ13:head : candidate had a stat error' 2026-02-17T12:02:40.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1172: TEST_corrupt_scrub_replicated: err_strings[13]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 3:f2a5b2a4:::ROBJ3:head : missing' 2026-02-17T12:02:40.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1173: TEST_corrupt_scrub_replicated: err_strings[14]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ffdb2004:::ROBJ9:head : candidate size 1 info size 7 mismatch' 2026-02-17T12:02:40.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1174: TEST_corrupt_scrub_replicated: err_strings[15]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ffdb2004:::ROBJ9:head : object info inconsistent ' 2026-02-17T12:02:40.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1175: TEST_corrupt_scrub_replicated: err_strings[16]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 3:c0c86b1d:::ROBJ14:head : no '\''_'\'' attr' 2026-02-17T12:02:40.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1176: TEST_corrupt_scrub_replicated: err_strings[17]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 3:5c7b2c47:::ROBJ16:head : can'\''t decode '\''snapset'\'' attr .* v=3 cannot decode .* Malformed input' 2026-02-17T12:02:40.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1177: TEST_corrupt_scrub_replicated: err_strings[18]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub : stat mismatch, got 19/19 objects, 0/0 clones, 18/19 dirty, 18/19 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 1049713/1049720 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' 2026-02-17T12:02:40.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1178: TEST_corrupt_scrub_replicated: err_strings[19]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 1 missing, 8 inconsistent objects' 2026-02-17T12:02:40.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1179: TEST_corrupt_scrub_replicated: err_strings[20]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 18 errors' 2026-02-17T12:02:40.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1180: TEST_corrupt_scrub_replicated: err_strings[21]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:123a5f55:::ROBJ19:head : size 1049600 > 1048576 is too large' 2026-02-17T12:02:40.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:30259878:::ROBJ15:head : candidate had a missing info key' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:33aca486:::ROBJ18:head : object info inconsistent ' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:5c7b2c47:::ROBJ16:head : candidate had a corrupt snapset' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:5c7b2c47:::ROBJ16:head : candidate had a missing snapset key' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:5c7b2c47:::ROBJ16:head : failed to pick suitable object info' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:86586531:::ROBJ8:head : attr value mismatch '\''_key1-ROBJ8'\'', attr name mismatch '\''_key3-ROBJ8'\'', attr name mismatch '\''_key2-ROBJ8'\''' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:bc819597:::ROBJ12:head : candidate had a stat error' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:c0c86b1d:::ROBJ14:head : candidate had a missing info key' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:c0c86b1d:::ROBJ14:head : candidate had a corrupt info' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:c0c86b1d:::ROBJ14:head : failed to pick suitable object info' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : candidate size 9 info size 7 mismatch' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : size 9 != size 7 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from shard 0' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:d60617f9:::ROBJ13:head : candidate had a stat error' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 3:f2a5b2a4:::ROBJ3:head : missing' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ffdb2004:::ROBJ9:head : candidate size 1 info size 7 mismatch' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ffdb2004:::ROBJ9:head : object info inconsistent ' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 3:c0c86b1d:::ROBJ14:head : no '\''_'\'' attr' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 3:5c7b2c47:::ROBJ16:head : can'\''t decode '\''snapset'\'' attr .* v=3 cannot decode .* Malformed input' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub : stat mismatch, got 19/19 objects, 0/0 clones, 18/19 dirty, 18/19 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 1049713/1049720 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 1 missing, 8 inconsistent objects' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 18 errors' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1182: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:02:40.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1184: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:123a5f55:::ROBJ19:head : size 1049600 > 1048576 is too large' td/osd-scrub-repair/osd.1.log 2026-02-17T12:02:40.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1191: TEST_corrupt_scrub_replicated: rados list-inconsistent-pg csr_pool 2026-02-17T12:02:40.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1193: TEST_corrupt_scrub_replicated: jq '. | length' td/osd-scrub-repair/json 2026-02-17T12:02:40.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1193: TEST_corrupt_scrub_replicated: test 1 = 1 2026-02-17T12:02:40.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1195: TEST_corrupt_scrub_replicated: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-17T12:02:40.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1195: TEST_corrupt_scrub_replicated: test 3.0 = 3.0 2026-02-17T12:02:40.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1197: TEST_corrupt_scrub_replicated: rados list-inconsistent-obj 3.0 2026-02-17T12:02:40.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1199: TEST_corrupt_scrub_replicated: jq .epoch td/osd-scrub-repair/json 2026-02-17T12:02:40.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1199: TEST_corrupt_scrub_replicated: epoch=181 2026-02-17T12:02:40.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1201: TEST_corrupt_scrub_replicated: jq 'def walk(f): 2026-02-17T12:02:40.255 INFO:tasks.workunit.client.0.trial128.stderr: . as $in 2026-02-17T12:02:40.255 INFO:tasks.workunit.client.0.trial128.stderr: | if type == "object" then 2026-02-17T12:02:40.255 INFO:tasks.workunit.client.0.trial128.stderr: reduce keys[] as $key 2026-02-17T12:02:40.255 INFO:tasks.workunit.client.0.trial128.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T12:02:40.255 INFO:tasks.workunit.client.0.trial128.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T12:02:40.255 INFO:tasks.workunit.client.0.trial128.stderr: else f 2026-02-17T12:02:40.255 INFO:tasks.workunit.client.0.trial128.stderr: end; 2026-02-17T12:02:40.255 INFO:tasks.workunit.client.0.trial128.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T12:02:40.255 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T12:02:40.255 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T12:02:40.255 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T12:02:40.256 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-17T12:02:40.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1201: TEST_corrupt_scrub_replicated: jq .inconsistents 2026-02-17T12:02:40.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:1201: TEST_corrupt_scrub_replicated: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T12:02:40.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2082: TEST_corrupt_scrub_replicated: jq 'def walk(f): 2026-02-17T12:02:40.274 INFO:tasks.workunit.client.0.trial128.stderr: . as $in 2026-02-17T12:02:40.274 INFO:tasks.workunit.client.0.trial128.stderr: | if type == "object" then 2026-02-17T12:02:40.274 INFO:tasks.workunit.client.0.trial128.stderr: reduce keys[] as $key 2026-02-17T12:02:40.274 INFO:tasks.workunit.client.0.trial128.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T12:02:40.274 INFO:tasks.workunit.client.0.trial128.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T12:02:40.275 INFO:tasks.workunit.client.0.trial128.stderr: else f 2026-02-17T12:02:40.275 INFO:tasks.workunit.client.0.trial128.stderr: end; 2026-02-17T12:02:40.275 INFO:tasks.workunit.client.0.trial128.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T12:02:40.275 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T12:02:40.275 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T12:02:40.275 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T12:02:40.275 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-17T12:02:40.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2082: TEST_corrupt_scrub_replicated: jq .inconsistents 2026-02-17T12:02:40.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2082: TEST_corrupt_scrub_replicated: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T12:02:40.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2083: TEST_corrupt_scrub_replicated: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T12:02:40.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T12:02:40.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2084: TEST_corrupt_scrub_replicated: test no = yes 2026-02-17T12:02:40.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2089: TEST_corrupt_scrub_replicated: test '' = yes 2026-02-17T12:02:40.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2094: TEST_corrupt_scrub_replicated: objname=ROBJ9 2026-02-17T12:02:40.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2096: TEST_corrupt_scrub_replicated: echo -n ZZZ 2026-02-17T12:02:40.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2097: TEST_corrupt_scrub_replicated: rados --pool csr_pool put ROBJ9 td/osd-scrub-repair/change 2026-02-17T12:02:40.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2099: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-17T12:02:40.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:02:40.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:02:40.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T12:02:40.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:02:40.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-17T12:02:40.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:02:40.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:02:40.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T12:02:40.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:02:40.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T12:02:40.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:02:40.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:02:40.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:02:40.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:02:40.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:02:41.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:02:41.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-17T12:02:41.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:02:41.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:02:41.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:02:41.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:02:41.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:02:41.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ9 set-attr _ td/osd-scrub-repair/robj9-oi 2026-02-17T12:02:42.260 INFO:tasks.workunit.client.0.trial128.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-17T12:02:42.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T12:02:42.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:02:42.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:02:42.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:02:42.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:02:42.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:02:42.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:02:42.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:02:42.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:02:42.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:02:42.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:02:42.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:02:42.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:02:42.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:02:42.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:02:42.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:02:42.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:42.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:42.796 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:42.797 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:42.797 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:02:42.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:02:42.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:02:42.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:02:42.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:02:42.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:02:42.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:02:42.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:02:42.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:02:42.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:02:42.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:02:42.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:02:42.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:02:42.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:02:42.799 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T12:02:42.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:02:42.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T12:02:42.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:02:42.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:02:42.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:02:42.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:02:42.822 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:42.815+0000 7fba7d0278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:42.823 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:42.815+0000 7fba7d0278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:42.825 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:42.815+0000 7fba7d0278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:42.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:02:42.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:02:42.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:02:42.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:02:42.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:02:42.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:42.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:02:42.955 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:02:42.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:42.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:43.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:44.067 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:44.059+0000 7fba7d0278c0 -1 Falling back to public interface 2026-02-17T12:02:44.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:44.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:44.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:02:44.110 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:02:44.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:44.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:44.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:45.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:45.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:45.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:02:45.262 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:02:45.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:45.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:45.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:46.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:46.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:46.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:02:46.412 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:02:46.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:46.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:46.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:47.040 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:47.031+0000 7fba7d0278c0 -1 osd.0 183 log_to_monitors true 2026-02-17T12:02:47.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:47.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:47.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:02:47.560 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:02:47.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:47.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:02:47.714 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 187 up_thru 187 down_at 184 last_clean_interval [170,183) [v2:127.0.0.1:6802/41264086,v1:127.0.0.1:6803/41264086] [v2:127.0.0.1:6804/41264086,v1:127.0.0.1:6805/41264086] exists,up 298610f2-1d9b-4f6c-b9da-00d205e834ad 2026-02-17T12:02:47.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:02:47.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:02:47.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:02:47.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:02:47.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:02:47.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:02:47.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:02:47.715 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:02:47.715 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:02:47.715 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:02:47.716 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:02:47.716 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:02:47.716 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:02:47.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:02:47.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:02:47.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:02:47.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:02:47.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:02:47.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:02:47.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:02:47.991 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:02:47.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:02:47.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:02:47.991 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:02:48.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=803158884354 2026-02-17T12:02:48.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 803158884354 2026-02-17T12:02:48.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-803158884354' 2026-02-17T12:02:48.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:02:48.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:02:48.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=777389080583 2026-02-17T12:02:48.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 777389080583 2026-02-17T12:02:48.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-803158884354 1-777389080583' 2026-02-17T12:02:48.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:02:48.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-803158884354 2026-02-17T12:02:48.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:02:48.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:02:48.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-803158884354 2026-02-17T12:02:48.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:02:48.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=803158884354 2026-02-17T12:02:48.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 803158884354' 2026-02-17T12:02:48.137 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 803158884354 2026-02-17T12:02:48.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:02:48.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 803158884354 2026-02-17T12:02:48.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:02:49.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:02:49.299 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:02:49.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 803158884354 2026-02-17T12:02:49.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:02:50.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:02:50.454 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:02:50.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 803158884354 -lt 803158884354 2026-02-17T12:02:50.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:02:50.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-777389080583 2026-02-17T12:02:50.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:02:50.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:02:50.612 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-777389080583 2026-02-17T12:02:50.612 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:02:50.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=777389080583 2026-02-17T12:02:50.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 777389080583' 2026-02-17T12:02:50.613 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 777389080583 2026-02-17T12:02:50.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:02:50.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 777389080584 -lt 777389080583 2026-02-17T12:02:50.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:02:50.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:02:50.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:02:50.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:02:50.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:02:50.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:02:50.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:02:50.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:02:50.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:02:50.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:02:50.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:02:51.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:02:51.114 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:02:51.114 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:02:51.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:02:51.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:02:51.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:02:51.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:02:51.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2100: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/oi td/osd-scrub-repair/change 2026-02-17T12:02:51.305 INFO:tasks.workunit.client.0.trial128.stderr:rm: cannot remove 'td/osd-scrub-repair/oi': No such file or directory 2026-02-17T12:02:51.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2102: TEST_corrupt_scrub_replicated: objname=ROBJ10 2026-02-17T12:02:51.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2103: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ10 get-attr _ 2026-02-17T12:02:51.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:02:51.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:02:51.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:02:51.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:02:51.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ10 get-attr _ 2026-02-17T12:02:51.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:02:51.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:02:51.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:02:51.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:02:51.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:02:51.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:02:51.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:02:51.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:02:51.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:02:51.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:02:52.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:02:52.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ10 get-attr _ 2026-02-17T12:02:52.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:02:52.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:02:52.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:02:52.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:02:52.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:02:52.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ10 get-attr _ 2026-02-17T12:02:52.924 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-17T12:02:52.924 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-17T12:02:53.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T12:02:53.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:02:53.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:02:53.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:02:53.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:02:53.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:02:53.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:02:53.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:02:53.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:02:53.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:02:53.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:02:53.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:02:53.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:02:53.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:02:53.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:02:53.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:02:53.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:02:53.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:02:53.459 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:02:53.459 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:02:53.459 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:02:53.460 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:02:53.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:02:53.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:02:53.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:02:53.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:02:53.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:02:53.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:02:53.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:02:53.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:02:53.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:02:53.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:02:53.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:02:53.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:02:53.462 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:02:53.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:02:53.463 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:02:53.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:02:53.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:02:53.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:02:53.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:02:53.480 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:53.471+0000 7f243e3978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:53.482 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:53.475+0000 7f243e3978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:53.482 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:53.475+0000 7f243e3978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:02:53.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:02:53.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:02:53.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:02:53.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:02:53.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:02:53.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:53.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:02:53.617 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:02:53.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:53.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:53.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:54.727 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:54.719+0000 7f243e3978c0 -1 Falling back to public interface 2026-02-17T12:02:54.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:54.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:54.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:02:54.768 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:02:54.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:54.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:54.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:55.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:55.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:55.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:02:55.916 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:02:55.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:55.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:56.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:57.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:57.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:57.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:02:57.067 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:02:57.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:57.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:57.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:02:57.708 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:02:57.699+0000 7f243e3978c0 -1 osd.1 188 log_to_monitors true 2026-02-17T12:02:58.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:02:58.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:02:58.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:02:58.219 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:02:58.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:02:58.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:02:58.365 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 192 up_thru 192 down_at 189 last_clean_interval [181,188) [v2:127.0.0.1:6810/3998498324,v1:127.0.0.1:6811/3998498324] [v2:127.0.0.1:6812/3998498324,v1:127.0.0.1:6813/3998498324] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T12:02:58.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:02:58.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:02:58.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:02:58.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:02:58.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:02:58.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:02:58.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:02:58.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:02:58.366 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:02:58.366 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:02:58.367 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:02:58.367 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:02:58.367 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:02:58.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:02:58.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:02:58.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:02:58.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:02:58.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:02:58.476 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:02:58.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:02:58.630 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:02:58.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:02:58.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:02:58.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:02:58.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=803158884357 2026-02-17T12:02:58.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 803158884357 2026-02-17T12:02:58.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-803158884357' 2026-02-17T12:02:58.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:02:58.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:02:58.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=824633720834 2026-02-17T12:02:58.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 824633720834 2026-02-17T12:02:58.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-803158884357 1-824633720834' 2026-02-17T12:02:58.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:02:58.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-803158884357 2026-02-17T12:02:58.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:02:58.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:02:58.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-803158884357 2026-02-17T12:02:58.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:02:58.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=803158884357 2026-02-17T12:02:58.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 803158884357' 2026-02-17T12:02:58.777 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 803158884357 2026-02-17T12:02:58.778 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:02:58.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 803158884356 -lt 803158884357 2026-02-17T12:02:58.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:02:59.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:02:59.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:03:00.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 803158884357 -lt 803158884357 2026-02-17T12:03:00.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:03:00.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-824633720834 2026-02-17T12:03:00.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:03:00.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:03:00.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-824633720834 2026-02-17T12:03:00.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:03:00.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=824633720834 2026-02-17T12:03:00.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 824633720834' 2026-02-17T12:03:00.104 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 824633720834 2026-02-17T12:03:00.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:03:00.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 824633720834 -lt 824633720834 2026-02-17T12:03:00.253 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:03:00.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:03:00.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:03:00.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:03:00.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:03:00.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:03:00.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:03:00.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:03:00.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:03:00.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:03:00.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:03:00.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:03:00.599 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:03:00.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:03:00.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:03:00.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:03:00.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:03:00.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:03:00.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2104: TEST_corrupt_scrub_replicated: rados --pool csr_pool setomapval ROBJ10 key2-ROBJ10 val2-ROBJ10 2026-02-17T12:03:00.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2105: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-17T12:03:00.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:03:00.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:03:00.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T12:03:00.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:03:00.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-17T12:03:00.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:03:00.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:03:00.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T12:03:00.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:03:00.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T12:03:00.807 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:00.807 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:00.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:00.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:00.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:02.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:02.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-17T12:03:02.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:03:02.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:03:02.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:03:02.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:03:02.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:03:02.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-17T12:03:02.772 INFO:tasks.workunit.client.0.trial128.stderr:Error decoding attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (22) Invalid argument 2026-02-17T12:03:03.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T12:03:03.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:03:03.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:03:03.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:03:03.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:03:03.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:03:03.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:03:03.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:03:03.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:03:03.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:03:03.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:03:03.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:03:03.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:03:03.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:03:03.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:03:03.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:03:03.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:03.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:03.308 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:03.308 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:03.308 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:03.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:03:03.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:03:03.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:03:03.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:03:03.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:03:03.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:03:03.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:03:03.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:03:03.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:03:03.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:03:03.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:03:03.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:03:03.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:03:03.310 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T12:03:03.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:03:03.311 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T12:03:03.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:03:03.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:03:03.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:03:03.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:03:03.333 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:03.323+0000 7fcb4483a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:03.335 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:03.327+0000 7fcb4483a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:03.336 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:03.327+0000 7fcb4483a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:03.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:03:03.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:03:03.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:03:03.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:03:03.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:03:03.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:03.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:03:03.469 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:03:03.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:03.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:03.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:04.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:04.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:04.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:03:04.620 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:03:04.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:04.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:04.623 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:04.615+0000 7fcb4483a8c0 -1 Falling back to public interface 2026-02-17T12:03:04.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:05.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:05.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:05.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:03:05.771 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:03:05.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:05.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:05.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:06.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:06.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:06.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:03:06.925 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:03:06.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:06.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:07.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:07.642 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:07.635+0000 7fcb4483a8c0 -1 osd.0 193 log_to_monitors true 2026-02-17T12:03:08.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:08.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:08.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:03:08.076 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:03:08.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:08.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:08.228 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 197 up_thru 197 down_at 194 last_clean_interval [187,193) [v2:127.0.0.1:6802/604139860,v1:127.0.0.1:6803/604139860] [v2:127.0.0.1:6804/604139860,v1:127.0.0.1:6805/604139860] exists,up 298610f2-1d9b-4f6c-b9da-00d205e834ad 2026-02-17T12:03:08.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:03:08.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:03:08.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:03:08.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:03:08.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:03:08.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:03:08.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:03:08.229 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:03:08.230 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:03:08.230 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:03:08.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:03:08.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:03:08.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:03:08.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:03:08.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:03:08.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:03:08.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:03:08.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:03:08.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:03:08.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:03:08.495 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:03:08.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:03:08.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:03:08.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:03:08.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=846108557314 2026-02-17T12:03:08.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 846108557314 2026-02-17T12:03:08.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-846108557314' 2026-02-17T12:03:08.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:03:08.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:03:08.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=824633720837 2026-02-17T12:03:08.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 824633720837 2026-02-17T12:03:08.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-846108557314 1-824633720837' 2026-02-17T12:03:08.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:03:08.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-846108557314 2026-02-17T12:03:08.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:03:08.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:03:08.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-846108557314 2026-02-17T12:03:08.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:03:08.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=846108557314 2026-02-17T12:03:08.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 846108557314' 2026-02-17T12:03:08.644 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 846108557314 2026-02-17T12:03:08.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:03:08.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 846108557314 2026-02-17T12:03:08.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:03:09.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:03:09.812 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:03:09.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 846108557314 -lt 846108557314 2026-02-17T12:03:09.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:03:09.964 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-824633720837 2026-02-17T12:03:09.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:03:09.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:03:09.967 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-824633720837 2026-02-17T12:03:09.967 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:03:09.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=824633720837 2026-02-17T12:03:09.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 824633720837' 2026-02-17T12:03:09.969 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 824633720837 2026-02-17T12:03:09.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:03:10.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 824633720837 -lt 824633720837 2026-02-17T12:03:10.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:03:10.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:03:10.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:03:10.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:03:10.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:03:10.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:03:10.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:03:10.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:03:10.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:03:10.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:03:10.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:03:10.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:03:10.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:03:10.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:03:10.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:03:10.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:03:10.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:03:10.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:03:10.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2106: TEST_corrupt_scrub_replicated: objectstore_tool td/osd-scrub-repair 1 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-17T12:03:10.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:03:10.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:03:10.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:03:10.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:03:10.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-17T12:03:10.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:03:10.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:03:10.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:03:10.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:03:10.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:03:10.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:10.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:10.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:10.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:10.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:11.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:11.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-17T12:03:11.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:03:11.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:03:11.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:03:11.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:03:11.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:03:11.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 ROBJ10 set-attr _ td/osd-scrub-repair/oi 2026-02-17T12:03:12.628 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 3.0_head,#3:30259878:::ROBJ15:head#, (61) No data available 2026-02-17T12:03:12.628 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 3.0_head,#3:c0c86b1d:::ROBJ14:head#, (61) No data available 2026-02-17T12:03:13.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T12:03:13.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:03:13.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:03:13.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:03:13.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:03:13.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:03:13.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:03:13.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:03:13.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:03:13.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:03:13.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:03:13.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:03:13.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:03:13.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:03:13.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:03:13.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:03:13.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:13.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:13.172 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:13.172 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:13.173 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:13.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:03:13.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:03:13.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:03:13.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:03:13.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:03:13.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:03:13.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:03:13.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:03:13.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:03:13.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:03:13.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:03:13.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:03:13.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:03:13.175 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:03:13.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:03:13.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:03:13.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:03:13.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:03:13.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:03:13.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:03:13.190 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:13.183+0000 7f025e6fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:13.191 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:13.183+0000 7f025e6fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:13.192 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:13.183+0000 7f025e6fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:13.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:03:13.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:03:13.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:03:13.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:03:13.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:03:13.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:13.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:03:13.324 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:03:13.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:13.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:03:13.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:14.439 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:14.431+0000 7f025e6fc8c0 -1 Falling back to public interface 2026-02-17T12:03:14.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:14.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:14.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:03:14.483 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:03:14.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:14.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:03:14.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:15.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:15.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:15.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:03:15.641 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:03:15.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:15.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:03:15.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:16.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:16.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:16.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:03:16.795 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:03:16.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:16.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:03:16.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:17.422 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:17.415+0000 7f025e6fc8c0 -1 osd.1 198 log_to_monitors true 2026-02-17T12:03:17.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:17.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:17.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:03:17.952 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:03:17.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:17.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:03:18.107 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 202 up_thru 202 down_at 199 last_clean_interval [192,198) [v2:127.0.0.1:6810/2966795743,v1:127.0.0.1:6811/2966795743] [v2:127.0.0.1:6812/2966795743,v1:127.0.0.1:6813/2966795743] exists,up e810ce24-5a23-4965-9d34-3976c6e1a6f8 2026-02-17T12:03:18.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:03:18.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:03:18.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:03:18.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:03:18.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:03:18.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:03:18.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:03:18.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:03:18.109 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:03:18.109 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:03:18.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:03:18.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:03:18.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:03:18.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:03:18.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:03:18.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:03:18.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:03:18.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:03:18.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:03:18.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:03:18.386 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:03:18.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:03:18.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:03:18.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:03:18.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=846108557317 2026-02-17T12:03:18.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 846108557317 2026-02-17T12:03:18.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-846108557317' 2026-02-17T12:03:18.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:03:18.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:03:18.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=867583393794 2026-02-17T12:03:18.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 867583393794 2026-02-17T12:03:18.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-846108557317 1-867583393794' 2026-02-17T12:03:18.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:03:18.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-846108557317 2026-02-17T12:03:18.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:03:18.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:03:18.534 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-846108557317 2026-02-17T12:03:18.534 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:03:18.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=846108557317 2026-02-17T12:03:18.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 846108557317' 2026-02-17T12:03:18.536 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 846108557317 2026-02-17T12:03:18.536 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:03:18.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 846108557316 -lt 846108557317 2026-02-17T12:03:18.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:03:19.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:03:19.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:03:19.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 846108557316 -lt 846108557317 2026-02-17T12:03:19.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:03:20.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:03:20.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:03:21.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 846108557317 -lt 846108557317 2026-02-17T12:03:21.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:03:21.007 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-867583393794 2026-02-17T12:03:21.007 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:03:21.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:03:21.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-867583393794 2026-02-17T12:03:21.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:03:21.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=867583393794 2026-02-17T12:03:21.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 867583393794' 2026-02-17T12:03:21.012 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 867583393794 2026-02-17T12:03:21.012 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:03:21.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 867583393794 -lt 867583393794 2026-02-17T12:03:21.169 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:03:21.170 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:03:21.170 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:03:21.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:03:21.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:03:21.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:03:21.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:03:21.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:03:21.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:03:21.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:03:21.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:03:21.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:03:21.509 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:03:21.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:03:21.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:03:21.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:03:21.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:03:21.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:03:21.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2107: TEST_corrupt_scrub_replicated: rm td/osd-scrub-repair/oi 2026-02-17T12:03:21.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2109: TEST_corrupt_scrub_replicated: inject_eio rep data csr_pool ROBJ11 td/osd-scrub-repair 0 2026-02-17T12:03:21.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T12:03:21.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:03:21.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:03:21.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:03:21.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=csr_pool 2026-02-17T12:03:21.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:03:21.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=ROBJ11 2026-02-17T12:03:21.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:03:21.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/osd-scrub-repair 2026-02-17T12:03:21.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:03:21.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-17T12:03:21.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:03:21.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds csr_pool ROBJ11 2026-02-17T12:03:21.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=csr_pool 2026-02-17T12:03:21.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=ROBJ11 2026-02-17T12:03:21.700 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map csr_pool ROBJ11 2026-02-17T12:03:21.700 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:03:21.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T12:03:21.852 INFO:tasks.workunit.client.0.trial128.stderr:0' 2026-02-17T12:03:21.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2026-02-17T12:03:21.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0') 2026-02-17T12:03:21.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:03:21.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=1 2026-02-17T12:03:21.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T12:03:21.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T12:03:21.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/osd-scrub-repair/1/type 2026-02-17T12:03:21.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:03:21.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-17T12:03:21.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:03:21.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T12:03:21.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:03:21.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:03:21.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:03:21.856 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T12:03:21.856 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:03:21.856 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:03:21.856 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:21.856 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:21.856 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:21.856 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.1.asok 2026-02-17T12:03:21.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40886/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:03:21.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:03:21.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:03:21.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:03:21.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.1 2026-02-17T12:03:21.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:03:21.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:03:21.919 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:21.920 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:21.920 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:21.920 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.1.asok 2026-02-17T12:03:21.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:03:21.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.40886/ceph-osd.1.asok injectdataerr csr_pool ROBJ11 2026-02-17T12:03:21.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2110: TEST_corrupt_scrub_replicated: inject_eio rep mdata csr_pool ROBJ12 td/osd-scrub-repair 1 2026-02-17T12:03:21.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T12:03:21.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:03:21.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=mdata 2026-02-17T12:03:21.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:03:21.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=csr_pool 2026-02-17T12:03:21.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:03:21.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=ROBJ12 2026-02-17T12:03:21.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:03:21.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/osd-scrub-repair 2026-02-17T12:03:21.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:03:21.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=1 2026-02-17T12:03:21.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:03:21.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds csr_pool ROBJ12 2026-02-17T12:03:21.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=csr_pool 2026-02-17T12:03:21.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=ROBJ12 2026-02-17T12:03:21.983 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map csr_pool ROBJ12 2026-02-17T12:03:21.983 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:03:22.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T12:03:22.133 INFO:tasks.workunit.client.0.trial128.stderr:0' 2026-02-17T12:03:22.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2026-02-17T12:03:22.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0') 2026-02-17T12:03:22.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:03:22.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=0 2026-02-17T12:03:22.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T12:03:22.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T12:03:22.134 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/osd-scrub-repair/0/type 2026-02-17T12:03:22.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:03:22.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-17T12:03:22.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:03:22.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=0 2026-02-17T12:03:22.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:03:22.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:03:22.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:03:22.136 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.0 2026-02-17T12:03:22.136 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:03:22.137 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:03:22.137 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:22.137 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:22.137 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:22.137 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.0.asok 2026-02-17T12:03:22.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40886/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:03:22.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:03:22.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:03:22.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:03:22.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.0 2026-02-17T12:03:22.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:03:22.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:03:22.200 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:22.201 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:22.201 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:22.201 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.0.asok 2026-02-17T12:03:22.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:03:22.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.40886/ceph-osd.0.asok injectmdataerr csr_pool ROBJ12 2026-02-17T12:03:22.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2111: TEST_corrupt_scrub_replicated: inject_eio rep mdata csr_pool ROBJ13 td/osd-scrub-repair 1 2026-02-17T12:03:22.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T12:03:22.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:03:22.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=mdata 2026-02-17T12:03:22.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:03:22.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=csr_pool 2026-02-17T12:03:22.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:03:22.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=ROBJ13 2026-02-17T12:03:22.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:03:22.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/osd-scrub-repair 2026-02-17T12:03:22.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:03:22.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=1 2026-02-17T12:03:22.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:03:22.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds csr_pool ROBJ13 2026-02-17T12:03:22.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=csr_pool 2026-02-17T12:03:22.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=ROBJ13 2026-02-17T12:03:22.263 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map csr_pool ROBJ13 2026-02-17T12:03:22.263 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:03:22.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T12:03:22.414 INFO:tasks.workunit.client.0.trial128.stderr:0' 2026-02-17T12:03:22.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2026-02-17T12:03:22.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0') 2026-02-17T12:03:22.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:03:22.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=0 2026-02-17T12:03:22.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T12:03:22.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T12:03:22.415 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/osd-scrub-repair/0/type 2026-02-17T12:03:22.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:03:22.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-17T12:03:22.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:03:22.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=0 2026-02-17T12:03:22.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:03:22.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:03:22.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:03:22.417 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.0 2026-02-17T12:03:22.417 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:03:22.418 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:03:22.418 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:22.418 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:22.418 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:22.418 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.0.asok 2026-02-17T12:03:22.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40886/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:03:22.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:03:22.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:03:22.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:03:22.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.0 2026-02-17T12:03:22.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:03:22.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:03:22.482 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:22.482 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:22.482 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:22.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.0.asok 2026-02-17T12:03:22.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:03:22.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.40886/ceph-osd.0.asok injectmdataerr csr_pool ROBJ13 2026-02-17T12:03:22.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2112: TEST_corrupt_scrub_replicated: inject_eio rep data csr_pool ROBJ13 td/osd-scrub-repair 0 2026-02-17T12:03:22.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-17T12:03:22.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-17T12:03:22.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-17T12:03:22.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-17T12:03:22.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=csr_pool 2026-02-17T12:03:22.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-17T12:03:22.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=ROBJ13 2026-02-17T12:03:22.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-17T12:03:22.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/osd-scrub-repair 2026-02-17T12:03:22.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-17T12:03:22.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-17T12:03:22.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-17T12:03:22.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds csr_pool ROBJ13 2026-02-17T12:03:22.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=csr_pool 2026-02-17T12:03:22.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=ROBJ13 2026-02-17T12:03:22.545 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map csr_pool ROBJ13 2026-02-17T12:03:22.545 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:03:22.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T12:03:22.707 INFO:tasks.workunit.client.0.trial128.stderr:0' 2026-02-17T12:03:22.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2026-02-17T12:03:22.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0') 2026-02-17T12:03:22.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-17T12:03:22.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=1 2026-02-17T12:03:22.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-17T12:03:22.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-17T12:03:22.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/osd-scrub-repair/1/type 2026-02-17T12:03:22.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-17T12:03:22.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-17T12:03:22.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-17T12:03:22.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-17T12:03:22.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-17T12:03:22.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-17T12:03:22.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-17T12:03:22.711 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-17T12:03:22.711 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:03:22.711 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:03:22.711 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:22.711 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:22.711 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:22.711 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.1.asok 2026-02-17T12:03:22.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.40886/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-17T12:03:22.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-17T12:03:22.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-17T12:03:22.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-17T12:03:22.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.1 2026-02-17T12:03:22.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:03:22.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:03:22.772 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:22.772 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:22.772 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:22.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-osd.1.asok 2026-02-17T12:03:22.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-17T12:03:22.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.40886/ceph-osd.1.asok injectdataerr csr_pool ROBJ13 2026-02-17T12:03:22.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2115: TEST_corrupt_scrub_replicated: ceph tell 'osd.*' injectargs -- --osd-max-object-size=134217728 2026-02-17T12:03:22.898 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: {} 2026-02-17T12:03:22.899 INFO:tasks.workunit.client.0.trial128.stderr:osd.0: osd_max_object_size = '' 2026-02-17T12:03:22.907 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: {} 2026-02-17T12:03:22.907 INFO:tasks.workunit.client.0.trial128.stderr:osd.1: osd_max_object_size = '' 2026-02-17T12:03:22.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2117: TEST_corrupt_scrub_replicated: pg_deep_scrub 3.0 2026-02-17T12:03:22.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: pg_deep_scrub: local pgid=3.0 2026-02-17T12:03:22.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: pg_deep_scrub: wait_for_pg_clean 3.0 2026-02-17T12:03:22.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=3.0 2026-02-17T12:03:22.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T12:03:22.916 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:03:22.916 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:03:22.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:03:22.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:03:22.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:03:23.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T12:03:23.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T12:03:23.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T12:03:23.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T12:03:23.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:03:23.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:03:23.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:03:23.363 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:03:23.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:03:23.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:03:23.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:03:23.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=846108557319 2026-02-17T12:03:23.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 846108557319 2026-02-17T12:03:23.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-846108557319' 2026-02-17T12:03:23.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:03:23.437 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:03:23.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=867583393796 2026-02-17T12:03:23.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 867583393796 2026-02-17T12:03:23.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-846108557319 1-867583393796' 2026-02-17T12:03:23.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:03:23.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-846108557319 2026-02-17T12:03:23.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:03:23.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:03:23.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-846108557319 2026-02-17T12:03:23.511 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:03:23.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=846108557319 2026-02-17T12:03:23.512 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 846108557319 2026-02-17T12:03:23.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 846108557319' 2026-02-17T12:03:23.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:03:23.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 846108557317 -lt 846108557319 2026-02-17T12:03:23.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:03:24.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:03:24.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:03:24.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 846108557319 -lt 846108557319 2026-02-17T12:03:24.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:03:24.832 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-867583393796 2026-02-17T12:03:24.832 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:03:24.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:03:24.834 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-867583393796 2026-02-17T12:03:24.834 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:03:24.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=867583393796 2026-02-17T12:03:24.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 867583393796' 2026-02-17T12:03:24.836 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 867583393796 2026-02-17T12:03:24.837 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:03:24.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 867583393796 -lt 867583393796 2026-02-17T12:03:24.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T12:03:24.991 INFO:tasks.workunit.client.0.trial128.stdout:#---------- 3.0 loop 0 2026-02-17T12:03:24.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-17T12:03:24.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 3.0 2026-02-17T12:03:24.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=3.0 2026-02-17T12:03:24.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T12:03:24.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 3.0 query 2026-02-17T12:03:24.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T12:03:25.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T12:03:25.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T12:03:25.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T12:03:25.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T12:03:25.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T12:03:25.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:03:25.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:03:25.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:03:25.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T12:03:25.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: local last_scrub=2026-02-17T11:58:47.837225+0000 2026-02-17T12:03:25.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: pg_deep_scrub: ceph pg deep-scrub 3.0 2026-02-17T12:03:25.360 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 3.0 on osd.1 to deep-scrub 2026-02-17T12:03:25.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: pg_deep_scrub: wait_for_scrub 3.0 2026-02-17T11:58:47.837225+0000 last_deep_scrub_stamp 2026-02-17T12:03:25.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T12:03:25.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T11:58:47.837225+0000 2026-02-17T12:03:25.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-17T12:03:25.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:03:25.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:03:25.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T12:03:25.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:03:25.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:03:25.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:03:25.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T12:03:25.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:58:47.837225+0000 '>' 2026-02-17T11:58:47.837225+0000 2026-02-17T12:03:25.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:03:26.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:03:26.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:03:26.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T12:03:26.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:03:26.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:03:26.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:03:26.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T12:03:26.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:58:47.837225+0000 '>' 2026-02-17T11:58:47.837225+0000 2026-02-17T12:03:26.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:03:27.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:03:27.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:03:27.683 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T12:03:27.683 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:03:27.683 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:03:27.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T12:03:27.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:03:27.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T11:58:47.837225+0000 '>' 2026-02-17T11:58:47.837225+0000 2026-02-17T12:03:27.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:03:28.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:03:28.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:03:28.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_deep_scrub_stamp 2026-02-17T12:03:28.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:03:28.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:03:28.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:03:28.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_deep_scrub_stamp' 2026-02-17T12:03:28.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:03:26.295802+0000 '>' 2026-02-17T11:58:47.837225+0000 2026-02-17T12:03:28.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:03:28.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2119: TEST_corrupt_scrub_replicated: err_strings=() 2026-02-17T12:03:29.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2120: TEST_corrupt_scrub_replicated: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:30259878:::ROBJ15:head : candidate had a missing info key' 2026-02-17T12:03:29.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2121: TEST_corrupt_scrub_replicated: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:33aca486:::ROBJ18:head : data_digest 0xbd89c912 != data_digest 0x2ddbf8f5 from auth oi 3:33aca486:::ROBJ18:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 54 dd 2ddbf8f5 od ddc3680f alloc_hint [[]0 0 255[]][)], object info inconsistent ' 2026-02-17T12:03:29.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2122: TEST_corrupt_scrub_replicated: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:33aca486:::ROBJ18:head : data_digest 0xbd89c912 != data_digest 0x2ddbf8f5 from auth oi 3:33aca486:::ROBJ18:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 54 dd 2ddbf8f5 od ddc3680f alloc_hint [[]0 0 255[]][)]' 2026-02-17T12:03:29.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2123: TEST_corrupt_scrub_replicated: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:33aca486:::ROBJ18:head : failed to pick suitable auth object' 2026-02-17T12:03:29.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2124: TEST_corrupt_scrub_replicated: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:5c7b2c47:::ROBJ16:head : candidate had a corrupt snapset' 2026-02-17T12:03:29.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2125: TEST_corrupt_scrub_replicated: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:5c7b2c47:::ROBJ16:head : candidate had a missing snapset key' 2026-02-17T12:03:29.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2126: TEST_corrupt_scrub_replicated: err_strings[6]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:5c7b2c47:::ROBJ16:head : failed to pick suitable object info' 2026-02-17T12:03:29.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2127: TEST_corrupt_scrub_replicated: err_strings[7]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:86586531:::ROBJ8:head : attr value mismatch '\''_key1-ROBJ8'\'', attr name mismatch '\''_key3-ROBJ8'\'', attr name mismatch '\''_key2-ROBJ8'\''' 2026-02-17T12:03:29.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2128: TEST_corrupt_scrub_replicated: err_strings[8]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:87abbf36:::ROBJ11:head : candidate had a read error' 2026-02-17T12:03:29.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2129: TEST_corrupt_scrub_replicated: err_strings[9]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:8aa5320e:::ROBJ17:head : data_digest 0x5af0c3ef != data_digest 0x2ddbf8f5 from auth oi 3:8aa5320e:::ROBJ17:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 51 dd 2ddbf8f5 od e9572720 alloc_hint [[]0 0 0[]][)]' 2026-02-17T12:03:29.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2130: TEST_corrupt_scrub_replicated: err_strings[10]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:8aa5320e:::ROBJ17:head : data_digest 0x5af0c3ef != data_digest 0x2ddbf8f5 from auth oi 3:8aa5320e:::ROBJ17:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 51 dd 2ddbf8f5 od e9572720 alloc_hint [[]0 0 0[]][)]' 2026-02-17T12:03:29.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2131: TEST_corrupt_scrub_replicated: err_strings[11]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:8aa5320e:::ROBJ17:head : failed to pick suitable auth object' 2026-02-17T12:03:29.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2132: TEST_corrupt_scrub_replicated: err_strings[12]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:8b55fa4b:::ROBJ7:head : omap_digest 0xefced57a != omap_digest 0x6a73cc07 from shard 1' 2026-02-17T12:03:29.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2133: TEST_corrupt_scrub_replicated: err_strings[13]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:8b55fa4b:::ROBJ7:head : omap_digest 0x6a73cc07 != omap_digest 0xefced57a from auth oi 3:8b55fa4b:::ROBJ7:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 21 dd 2ddbf8f5 od efced57a alloc_hint [[]0 0 0[]][)]' 2026-02-17T12:03:29.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2134: TEST_corrupt_scrub_replicated: err_strings[14]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:a53c12e8:::ROBJ6:head : omap_digest 0x689ee887 != omap_digest 0x179c919f from shard 1, omap_digest 0x689ee887 != omap_digest 0x179c919f from auth oi 3:a53c12e8:::ROBJ6:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 18 dd 2ddbf8f5 od 179c919f alloc_hint [[]0 0 0[]][)]' 2026-02-17T12:03:29.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2135: TEST_corrupt_scrub_replicated: err_strings[15]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:b1f19cbd:::ROBJ10:head : omap_digest 0xa8dd5adc != omap_digest 0xc2025a24 from auth oi 3:b1f19cbd:::ROBJ10:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 30 dd 2ddbf8f5 od c2025a24 alloc_hint [[]0 0 0[]][)]' 2026-02-17T12:03:29.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2136: TEST_corrupt_scrub_replicated: err_strings[16]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:b1f19cbd:::ROBJ10:head : omap_digest 0xa8dd5adc != omap_digest 0xc2025a24 from auth oi 3:b1f19cbd:::ROBJ10:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 30 dd 2ddbf8f5 od c2025a24 alloc_hint [[]0 0 0[]][)]' 2026-02-17T12:03:29.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2137: TEST_corrupt_scrub_replicated: err_strings[17]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:b1f19cbd:::ROBJ10:head : failed to pick suitable auth object' 2026-02-17T12:03:29.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2138: TEST_corrupt_scrub_replicated: err_strings[18]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:bc819597:::ROBJ12:head : candidate had a stat error' 2026-02-17T12:03:29.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2139: TEST_corrupt_scrub_replicated: err_strings[19]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:c0c86b1d:::ROBJ14:head : candidate had a missing info key' 2026-02-17T12:03:29.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2140: TEST_corrupt_scrub_replicated: err_strings[20]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:c0c86b1d:::ROBJ14:head : candidate had a corrupt info' 2026-02-17T12:03:29.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2141: TEST_corrupt_scrub_replicated: err_strings[21]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:c0c86b1d:::ROBJ14:head : failed to pick suitable object info' 2026-02-17T12:03:29.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2142: TEST_corrupt_scrub_replicated: err_strings[22]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : candidate size 9 info size 7 mismatch' 2026-02-17T12:03:29.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2143: TEST_corrupt_scrub_replicated: err_strings[23]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : data_digest 0x2d4a11c2 != data_digest 0x2ddbf8f5 from shard 0, data_digest 0x2d4a11c2 != data_digest 0x2ddbf8f5 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from shard 0' 2026-02-17T12:03:29.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2144: TEST_corrupt_scrub_replicated: err_strings[24]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:d60617f9:::ROBJ13:head : candidate had a read error' 2026-02-17T12:03:29.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2145: TEST_corrupt_scrub_replicated: err_strings[25]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:d60617f9:::ROBJ13:head : candidate had a stat error' 2026-02-17T12:03:29.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2146: TEST_corrupt_scrub_replicated: err_strings[26]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:d60617f9:::ROBJ13:head : failed to pick suitable object info' 2026-02-17T12:03:29.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2147: TEST_corrupt_scrub_replicated: err_strings[27]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:e97ce31e:::ROBJ2:head : data_digest 0x578a4830 != data_digest 0x2ddbf8f5 from shard 1, data_digest 0x578a4830 != data_digest 0x2ddbf8f5 from auth oi 3:e97ce31e:::ROBJ2:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 6 dd 2ddbf8f5 od f8e11918 alloc_hint [[]0 0 0[]][)]' 2026-02-17T12:03:29.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2148: TEST_corrupt_scrub_replicated: err_strings[28]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 3:f2a5b2a4:::ROBJ3:head : missing' 2026-02-17T12:03:29.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2149: TEST_corrupt_scrub_replicated: err_strings[29]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:f4981d31:::ROBJ4:head : omap_digest 0xd7178dfe != omap_digest 0xe2d46ea4 from shard 1, omap_digest 0xd7178dfe != omap_digest 0xe2d46ea4 from auth oi 3:f4981d31:::ROBJ4:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 12 dd 2ddbf8f5 od e2d46ea4 alloc_hint [[]0 0 0[]][)]' 2026-02-17T12:03:29.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2150: TEST_corrupt_scrub_replicated: err_strings[30]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:f4bfd4d1:::ROBJ5:head : omap_digest 0x1a862a41 != omap_digest 0x6cac8f6 from shard 1' 2026-02-17T12:03:29.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2151: TEST_corrupt_scrub_replicated: err_strings[31]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:f4bfd4d1:::ROBJ5:head : omap_digest 0x6cac8f6 != omap_digest 0x1a862a41 from auth oi 3:f4bfd4d1:::ROBJ5:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 15 dd 2ddbf8f5 od 1a862a41 alloc_hint [[]0 0 0[]][)]' 2026-02-17T12:03:29.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2152: TEST_corrupt_scrub_replicated: err_strings[32]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:ffdb2004:::ROBJ9:head : candidate size 3 info size 7 mismatch' 2026-02-17T12:03:29.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2153: TEST_corrupt_scrub_replicated: err_strings[33]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:ffdb2004:::ROBJ9:head : object info inconsistent ' 2026-02-17T12:03:29.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2154: TEST_corrupt_scrub_replicated: err_strings[34]='log_channel[(]cluster[)] log [[]ERR[]] : deep-scrub [0-9]*[.]0 3:c0c86b1d:::ROBJ14:head : no '\''_'\'' attr' 2026-02-17T12:03:29.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2155: TEST_corrupt_scrub_replicated: err_strings[35]='log_channel[(]cluster[)] log [[]ERR[]] : deep-scrub [0-9]*[.]0 3:5c7b2c47:::ROBJ16:head : can'\''t decode '\''snapset'\'' attr .* v=3 cannot decode .* Malformed input' 2026-02-17T12:03:29.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2156: TEST_corrupt_scrub_replicated: err_strings[36]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub : stat mismatch, got 19/19 objects, 0/0 clones, 18/19 dirty, 18/19 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 1049715/1049716 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' 2026-02-17T12:03:29.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2157: TEST_corrupt_scrub_replicated: err_strings[37]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub 1 missing, 11 inconsistent objects' 2026-02-17T12:03:29.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2158: TEST_corrupt_scrub_replicated: err_strings[38]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub 35 errors' 2026-02-17T12:03:29.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:30259878:::ROBJ15:head : candidate had a missing info key' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:33aca486:::ROBJ18:head : data_digest 0xbd89c912 != data_digest 0x2ddbf8f5 from auth oi 3:33aca486:::ROBJ18:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 54 dd 2ddbf8f5 od ddc3680f alloc_hint [[]0 0 255[]][)], object info inconsistent ' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:33aca486:::ROBJ18:head : data_digest 0xbd89c912 != data_digest 0x2ddbf8f5 from auth oi 3:33aca486:::ROBJ18:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 54 dd 2ddbf8f5 od ddc3680f alloc_hint [[]0 0 255[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:33aca486:::ROBJ18:head : failed to pick suitable auth object' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:5c7b2c47:::ROBJ16:head : candidate had a corrupt snapset' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:5c7b2c47:::ROBJ16:head : candidate had a missing snapset key' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:5c7b2c47:::ROBJ16:head : failed to pick suitable object info' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:86586531:::ROBJ8:head : attr value mismatch '\''_key1-ROBJ8'\'', attr name mismatch '\''_key3-ROBJ8'\'', attr name mismatch '\''_key2-ROBJ8'\''' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:87abbf36:::ROBJ11:head : candidate had a read error' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:8aa5320e:::ROBJ17:head : data_digest 0x5af0c3ef != data_digest 0x2ddbf8f5 from auth oi 3:8aa5320e:::ROBJ17:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 51 dd 2ddbf8f5 od e9572720 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:8aa5320e:::ROBJ17:head : data_digest 0x5af0c3ef != data_digest 0x2ddbf8f5 from auth oi 3:8aa5320e:::ROBJ17:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 51 dd 2ddbf8f5 od e9572720 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:8aa5320e:::ROBJ17:head : failed to pick suitable auth object' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:8b55fa4b:::ROBJ7:head : omap_digest 0xefced57a != omap_digest 0x6a73cc07 from shard 1' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:8b55fa4b:::ROBJ7:head : omap_digest 0x6a73cc07 != omap_digest 0xefced57a from auth oi 3:8b55fa4b:::ROBJ7:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 21 dd 2ddbf8f5 od efced57a alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:a53c12e8:::ROBJ6:head : omap_digest 0x689ee887 != omap_digest 0x179c919f from shard 1, omap_digest 0x689ee887 != omap_digest 0x179c919f from auth oi 3:a53c12e8:::ROBJ6:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 18 dd 2ddbf8f5 od 179c919f alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:b1f19cbd:::ROBJ10:head : omap_digest 0xa8dd5adc != omap_digest 0xc2025a24 from auth oi 3:b1f19cbd:::ROBJ10:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 30 dd 2ddbf8f5 od c2025a24 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:b1f19cbd:::ROBJ10:head : omap_digest 0xa8dd5adc != omap_digest 0xc2025a24 from auth oi 3:b1f19cbd:::ROBJ10:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 30 dd 2ddbf8f5 od c2025a24 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:b1f19cbd:::ROBJ10:head : failed to pick suitable auth object' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:bc819597:::ROBJ12:head : candidate had a stat error' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:c0c86b1d:::ROBJ14:head : candidate had a missing info key' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:c0c86b1d:::ROBJ14:head : candidate had a corrupt info' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:c0c86b1d:::ROBJ14:head : failed to pick suitable object info' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : candidate size 9 info size 7 mismatch' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:ce3f1d6a:::ROBJ1:head : data_digest 0x2d4a11c2 != data_digest 0x2ddbf8f5 from shard 0, data_digest 0x2d4a11c2 != data_digest 0x2ddbf8f5 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from auth oi 3:ce3f1d6a:::ROBJ1:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 3 dd 2ddbf8f5 od f5fba2c6 alloc_hint [[]0 0 0[]][)], size 9 != size 7 from shard 0' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:d60617f9:::ROBJ13:head : candidate had a read error' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:d60617f9:::ROBJ13:head : candidate had a stat error' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:d60617f9:::ROBJ13:head : failed to pick suitable object info' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:e97ce31e:::ROBJ2:head : data_digest 0x578a4830 != data_digest 0x2ddbf8f5 from shard 1, data_digest 0x578a4830 != data_digest 0x2ddbf8f5 from auth oi 3:e97ce31e:::ROBJ2:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 6 dd 2ddbf8f5 od f8e11918 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 3:f2a5b2a4:::ROBJ3:head : missing' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:f4981d31:::ROBJ4:head : omap_digest 0xd7178dfe != omap_digest 0xe2d46ea4 from shard 1, omap_digest 0xd7178dfe != omap_digest 0xe2d46ea4 from auth oi 3:f4981d31:::ROBJ4:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 12 dd 2ddbf8f5 od e2d46ea4 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid 3:f4bfd4d1:::ROBJ5:head : omap_digest 0x1a862a41 != omap_digest 0x6cac8f6 from shard 1' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 1 soid 3:f4bfd4d1:::ROBJ5:head : omap_digest 0x6cac8f6 != omap_digest 0x1a862a41 from auth oi 3:f4bfd4d1:::ROBJ5:head[(][0-9]*'\''[0-9]* osd.1.0:[0-9]* dirty|omap|data_digest|omap_digest s 7 uv 15 dd 2ddbf8f5 od 1a862a41 alloc_hint [[]0 0 0[]][)]' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:ffdb2004:::ROBJ9:head : candidate size 3 info size 7 mismatch' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard 0 soid 3:ffdb2004:::ROBJ9:head : object info inconsistent ' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : deep-scrub [0-9]*[.]0 3:c0c86b1d:::ROBJ14:head : no '\''_'\'' attr' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : deep-scrub [0-9]*[.]0 3:5c7b2c47:::ROBJ16:head : can'\''t decode '\''snapset'\'' attr .* v=3 cannot decode .* Malformed input' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub : stat mismatch, got 19/19 objects, 0/0 clones, 18/19 dirty, 18/19 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 1049715/1049716 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub 1 missing, 11 inconsistent objects' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2160: TEST_corrupt_scrub_replicated: for err_string in "${err_strings[@]}" 2026-02-17T12:03:29.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2162: TEST_corrupt_scrub_replicated: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 deep-scrub 35 errors' td/osd-scrub-repair/osd.1.log 2026-02-17T12:03:29.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2169: TEST_corrupt_scrub_replicated: rados list-inconsistent-pg csr_pool 2026-02-17T12:03:29.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2171: TEST_corrupt_scrub_replicated: jq '. | length' td/osd-scrub-repair/json 2026-02-17T12:03:29.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2171: TEST_corrupt_scrub_replicated: test 1 = 1 2026-02-17T12:03:29.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2173: TEST_corrupt_scrub_replicated: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-17T12:03:29.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2173: TEST_corrupt_scrub_replicated: test 3.0 = 3.0 2026-02-17T12:03:29.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2175: TEST_corrupt_scrub_replicated: rados list-inconsistent-obj 3.0 2026-02-17T12:03:29.374 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2177: TEST_corrupt_scrub_replicated: jq .epoch td/osd-scrub-repair/json 2026-02-17T12:03:29.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2177: TEST_corrupt_scrub_replicated: epoch=202 2026-02-17T12:03:29.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2179: TEST_corrupt_scrub_replicated: jq 'def walk(f): 2026-02-17T12:03:29.388 INFO:tasks.workunit.client.0.trial128.stderr: . as $in 2026-02-17T12:03:29.388 INFO:tasks.workunit.client.0.trial128.stderr: | if type == "object" then 2026-02-17T12:03:29.388 INFO:tasks.workunit.client.0.trial128.stderr: reduce keys[] as $key 2026-02-17T12:03:29.388 INFO:tasks.workunit.client.0.trial128.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T12:03:29.388 INFO:tasks.workunit.client.0.trial128.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T12:03:29.388 INFO:tasks.workunit.client.0.trial128.stderr: else f 2026-02-17T12:03:29.388 INFO:tasks.workunit.client.0.trial128.stderr: end; 2026-02-17T12:03:29.388 INFO:tasks.workunit.client.0.trial128.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T12:03:29.388 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T12:03:29.388 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T12:03:29.389 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T12:03:29.389 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-17T12:03:29.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2179: TEST_corrupt_scrub_replicated: jq .inconsistents 2026-02-17T12:03:29.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:2179: TEST_corrupt_scrub_replicated: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T12:03:29.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3558: TEST_corrupt_scrub_replicated: jq 'def walk(f): 2026-02-17T12:03:29.413 INFO:tasks.workunit.client.0.trial128.stderr: . as $in 2026-02-17T12:03:29.413 INFO:tasks.workunit.client.0.trial128.stderr: | if type == "object" then 2026-02-17T12:03:29.413 INFO:tasks.workunit.client.0.trial128.stderr: reduce keys[] as $key 2026-02-17T12:03:29.413 INFO:tasks.workunit.client.0.trial128.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T12:03:29.413 INFO:tasks.workunit.client.0.trial128.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T12:03:29.413 INFO:tasks.workunit.client.0.trial128.stderr: else f 2026-02-17T12:03:29.413 INFO:tasks.workunit.client.0.trial128.stderr: end; 2026-02-17T12:03:29.414 INFO:tasks.workunit.client.0.trial128.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T12:03:29.414 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T12:03:29.414 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T12:03:29.414 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T12:03:29.414 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-17T12:03:29.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3558: TEST_corrupt_scrub_replicated: jq .inconsistents 2026-02-17T12:03:29.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3558: TEST_corrupt_scrub_replicated: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T12:03:29.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3559: TEST_corrupt_scrub_replicated: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T12:03:29.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T12:03:29.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3560: TEST_corrupt_scrub_replicated: test no = yes 2026-02-17T12:03:29.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3565: TEST_corrupt_scrub_replicated: test '' = yes 2026-02-17T12:03:29.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3570: TEST_corrupt_scrub_replicated: repair 3.0 2026-02-17T12:03:29.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=3.0 2026-02-17T12:03:29.437 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 3.0 2026-02-17T12:03:29.437 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:03:29.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:03:29.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:03:29.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:03:29.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T12:03:26.295802+0000 2026-02-17T12:03:29.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 3.0 2026-02-17T12:03:29.727 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 3.0 on osd.1 to repair 2026-02-17T12:03:29.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 3.0 2026-02-17T12:03:26.295802+0000 2026-02-17T12:03:29.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T12:03:29.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:03:26.295802+0000 2026-02-17T12:03:29.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:03:29.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:03:29.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:03:29.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T12:03:29.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:03:29.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:03:29.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:03:29.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:03:29.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:03:26.295802+0000 '>' 2026-02-17T12:03:26.295802+0000 2026-02-17T12:03:29.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:03:30.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:03:30.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:03:30.892 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T12:03:30.892 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:03:30.892 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:03:30.893 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:03:30.893 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:03:31.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:03:26.295802+0000 '>' 2026-02-17T12:03:26.295802+0000 2026-02-17T12:03:31.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:03:32.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:03:32.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:03:32.049 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T12:03:32.049 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:03:32.049 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:03:32.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:03:32.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:03:32.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:03:26.295802+0000 '>' 2026-02-17T12:03:26.295802+0000 2026-02-17T12:03:32.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:03:33.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:03:33.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:03:33.205 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T12:03:33.205 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:03:33.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:03:33.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:03:33.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:03:33.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:03:26.295802+0000 '>' 2026-02-17T12:03:26.295802+0000 2026-02-17T12:03:33.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:03:34.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:03:34.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:03:34.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T12:03:34.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:03:34.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:03:34.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:03:34.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:03:34.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:03:30.342484+0000 '>' 2026-02-17T12:03:26.295802+0000 2026-02-17T12:03:34.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:03:34.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3571: TEST_corrupt_scrub_replicated: wait_for_clean 2026-02-17T12:03:34.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:03:34.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:03:34.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:03:34.521 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:03:34.521 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:03:34.521 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:03:34.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:03:34.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:03:34.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:03:34.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:03:34.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:03:34.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:03:34.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:03:34.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:03:34.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:03:34.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:03:34.786 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:03:34.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:03:34.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:03:34.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:03:34.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=846108557322 2026-02-17T12:03:34.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 846108557322 2026-02-17T12:03:34.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-846108557322' 2026-02-17T12:03:34.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:03:34.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:03:34.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=867583393799 2026-02-17T12:03:34.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 867583393799 2026-02-17T12:03:34.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-846108557322 1-867583393799' 2026-02-17T12:03:34.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:03:34.931 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-846108557322 2026-02-17T12:03:34.931 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:03:34.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:03:34.934 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-846108557322 2026-02-17T12:03:34.934 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:03:34.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=846108557322 2026-02-17T12:03:34.936 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 846108557322 2026-02-17T12:03:34.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 846108557322' 2026-02-17T12:03:34.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:03:35.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 846108557321 -lt 846108557322 2026-02-17T12:03:35.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:03:36.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:03:36.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:03:36.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 846108557322 -lt 846108557322 2026-02-17T12:03:36.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:03:36.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-867583393799 2026-02-17T12:03:36.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:03:36.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:03:36.257 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-867583393799 2026-02-17T12:03:36.257 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:03:36.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=867583393799 2026-02-17T12:03:36.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 867583393799' 2026-02-17T12:03:36.259 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 867583393799 2026-02-17T12:03:36.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:03:36.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 867583393799 -lt 867583393799 2026-02-17T12:03:36.411 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:03:36.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:03:36.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:03:36.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:03:36.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:03:36.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:03:36.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:03:36.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:03:36.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:03:36.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:03:36.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:03:36.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:03:36.743 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:03:36.743 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:03:36.743 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:03:36.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:03:36.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:03:36.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:03:36.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3574: TEST_corrupt_scrub_replicated: timeout 30 rados -p csr_pool get ROBJ17 td/osd-scrub-repair/robj17.out 2026-02-17T12:03:36.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3575: TEST_corrupt_scrub_replicated: timeout 30 rados -p csr_pool get ROBJ18 td/osd-scrub-repair/robj18.out 2026-02-17T12:03:36.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3577: TEST_corrupt_scrub_replicated: diff -q td/osd-scrub-repair/new.ROBJ17 td/osd-scrub-repair/robj17.out 2026-02-17T12:03:36.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3578: TEST_corrupt_scrub_replicated: rm -f td/osd-scrub-repair/new.ROBJ17 td/osd-scrub-repair/robj17.out 2026-02-17T12:03:36.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3579: TEST_corrupt_scrub_replicated: diff -q td/osd-scrub-repair/new.ROBJ18 td/osd-scrub-repair/robj18.out 2026-02-17T12:03:36.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3580: TEST_corrupt_scrub_replicated: rm -f td/osd-scrub-repair/new.ROBJ18 td/osd-scrub-repair/robj18.out 2026-02-17T12:03:36.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3582: TEST_corrupt_scrub_replicated: '[' 0 '!=' 0 ']' 2026-02-17T12:03:36.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:3588: TEST_corrupt_scrub_replicated: ceph osd pool rm csr_pool csr_pool --yes-i-really-really-mean-it 2026-02-17T12:03:37.166 INFO:tasks.workunit.client.0.trial128.stderr:pool 'csr_pool' removed 2026-02-17T12:03:37.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T12:03:37.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:03:37.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:03:37.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:03:37.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:37.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:37.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:37.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:37.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:37.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:37.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:03:37.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:03:37.292 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:03:37.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:03:37.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:03:37.294 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:03:37.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:37.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:03:37.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:03:37.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:37.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:03:37.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:03:37.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:03:37.333 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:03:37.333 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:37.333 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:37.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:03:37.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:03:37.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:03:37.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T12:03:37.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T12:03:37.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T12:03:37.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T12:03:37.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:03:37.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:03:37.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:03:37.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:03:37.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:03:37.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:03:37.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:03:37.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:03:37.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:03:37.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:03:37.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:03:37.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:03:37.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:03:37.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:03:37.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:03:37.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:37.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:03:37.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:03:37.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:03:37.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:03:37.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:03:37.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:03:37.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:03:37.349 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:37.349 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:37.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:03:37.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:03:37.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:03:37.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T12:03:37.351 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:03:37.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:37.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:37.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T12:03:37.353 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:03:37.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:03:37.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:03:37.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T12:03:37.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_corrupt_snapset_scrub_rep td/osd-scrub-repair 2026-02-17T12:03:37.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5931: TEST_corrupt_snapset_scrub_rep: local dir=td/osd-scrub-repair 2026-02-17T12:03:37.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5932: TEST_corrupt_snapset_scrub_rep: local poolname=csr_pool 2026-02-17T12:03:37.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5933: TEST_corrupt_snapset_scrub_rep: local total_objs=2 2026-02-17T12:03:37.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5935: TEST_corrupt_snapset_scrub_rep: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-17T12:03:37.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T12:03:37.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:03:37.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:03:37.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:03:37.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T12:03:37.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-17T12:03:37.377 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:03:37.377 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:37.377 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:37.377 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:37.377 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:37.377 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:37.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:03:37.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T12:03:37.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:03:37.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:03:37.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:03:37.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:03:37.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:03:37.411 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:03:37.411 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:03:37.411 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:37.411 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:37.411 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:37.411 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:37.412 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:37.412 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:03:37.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:03:37.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T12:03:37.464 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:03:37.464 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:03:37.464 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:03:37.464 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:03:37.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:03:37.465 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:03:37.465 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:03:37.465 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:03:37.466 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:03:37.466 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:37.466 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:37.466 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:03:37.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:03:37.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T12:03:37.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5936: TEST_corrupt_snapset_scrub_rep: run_mgr td/osd-scrub-repair x 2026-02-17T12:03:37.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T12:03:37.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:03:37.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:03:37.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:03:37.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T12:03:37.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:03:37.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:03:37.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:37.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:37.615 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:37.615 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:37.615 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:37.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:03:37.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:03:37.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:03:37.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5937: TEST_corrupt_snapset_scrub_rep: run_osd td/osd-scrub-repair 0 2026-02-17T12:03:37.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:03:37.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:03:37.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:03:37.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:03:37.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:03:37.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:03:37.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:03:37.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:03:37.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:03:37.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:03:37.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:03:37.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:03:37.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:03:37.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:03:37.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:03:37.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:37.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:37.639 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:37.639 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:37.639 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:37.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:03:37.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:03:37.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:03:37.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:03:37.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:03:37.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:03:37.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:03:37.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:03:37.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:03:37.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:03:37.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:03:37.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:03:37.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:03:37.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:03:37.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:03:37.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b6906687-b163-46f4-a29c-daa47d5e9644 2026-02-17T12:03:37.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 b6906687-b163-46f4-a29c-daa47d5e9644' 2026-02-17T12:03:37.643 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 b6906687-b163-46f4-a29c-daa47d5e9644 2026-02-17T12:03:37.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:03:37.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAZWZRpMYeQJhAAp/n9Y3ZWEn54qDi5ekgszQ== 2026-02-17T12:03:37.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAZWZRpMYeQJhAAp/n9Y3ZWEn54qDi5ekgszQ=="}' 2026-02-17T12:03:37.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b6906687-b163-46f4-a29c-daa47d5e9644 -i td/osd-scrub-repair/0/new.json 2026-02-17T12:03:37.743 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:03:37.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T12:03:37.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAZWZRpMYeQJhAAp/n9Y3ZWEn54qDi5ekgszQ== --osd-uuid b6906687-b163-46f4-a29c-daa47d5e9644 2026-02-17T12:03:37.767 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:37.758+0000 7fdd7fd598c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:37.769 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:37.762+0000 7fdd7fd598c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:37.770 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:37.762+0000 7fdd7fd598c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:37.770 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:37.762+0000 7fdd7fd598c0 -1 bdev(0x55b72f383c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:03:37.770 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:37.762+0000 7fdd7fd598c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T12:03:40.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T12:03:40.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:03:40.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:03:40.319 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:03:40.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:03:40.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:03:40.509 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:03:40.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:03:40.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:03:40.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:03:40.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:03:40.533 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:40.522+0000 7fd0f6ebe8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:40.535 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:40.526+0000 7fd0f6ebe8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:40.536 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:40.526+0000 7fd0f6ebe8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:40.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:03:40.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:03:40.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:03:40.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:03:40.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:03:40.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:40.669 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:03:40.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:03:40.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:40.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:40.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:41.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:41.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:41.822 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:03:41.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:03:41.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:41.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:41.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:42.051 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:42.042+0000 7fd0f6ebe8c0 -1 Falling back to public interface 2026-02-17T12:03:42.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:42.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:42.971 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:03:42.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:03:42.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:42.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:43.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:44.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:44.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:44.123 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:03:44.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:03:44.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:44.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:44.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:45.041 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:45.030+0000 7fd0f6ebe8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:03:45.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:45.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:45.273 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:03:45.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:03:45.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:45.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:45.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:46.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:46.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:46.425 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:03:46.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:03:46.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:46.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:03:46.573 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3363192272,v1:127.0.0.1:6803/3363192272] [v2:127.0.0.1:6804/3363192272,v1:127.0.0.1:6805/3363192272] exists,up b6906687-b163-46f4-a29c-daa47d5e9644 2026-02-17T12:03:46.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:03:46.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:03:46.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:03:46.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5938: TEST_corrupt_snapset_scrub_rep: run_osd td/osd-scrub-repair 1 2026-02-17T12:03:46.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:03:46.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:03:46.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:03:46.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:03:46.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:03:46.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:03:46.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:03:46.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:03:46.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:03:46.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:03:46.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:03:46.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:03:46.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:03:46.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:03:46.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:03:46.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:03:46.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:03:46.578 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:03:46.578 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:03:46.578 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:03:46.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:03:46.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:03:46.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:03:46.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:03:46.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:03:46.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:03:46.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:03:46.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:03:46.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:03:46.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:03:46.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:03:46.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:03:46.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:03:46.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:03:46.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:03:46.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b15e3214-4568-4115-a9ac-496330818a37 2026-02-17T12:03:46.580 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 b15e3214-4568-4115-a9ac-496330818a37 2026-02-17T12:03:46.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 b15e3214-4568-4115-a9ac-496330818a37' 2026-02-17T12:03:46.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:03:46.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAiWZRpjsX4IhAAzbny40PvAezSSjzKYzyazA== 2026-02-17T12:03:46.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAiWZRpjsX4IhAAzbny40PvAezSSjzKYzyazA=="}' 2026-02-17T12:03:46.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b15e3214-4568-4115-a9ac-496330818a37 -i td/osd-scrub-repair/1/new.json 2026-02-17T12:03:46.733 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:03:46.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T12:03:46.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAiWZRpjsX4IhAAzbny40PvAezSSjzKYzyazA== --osd-uuid b15e3214-4568-4115-a9ac-496330818a37 2026-02-17T12:03:46.759 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:46.750+0000 7f490fb2a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:46.760 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:46.750+0000 7f490fb2a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:46.761 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:46.754+0000 7f490fb2a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:46.762 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:46.754+0000 7f490fb2a8c0 -1 bdev(0x56383c827c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:03:46.762 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:46.754+0000 7f490fb2a8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T12:03:49.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T12:03:49.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:03:49.051 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:03:49.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:03:49.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:03:49.247 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:03:49.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:03:49.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:03:49.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:03:49.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:03:49.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:03:49.270 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:49.262+0000 7fc319d2d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:49.272 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:49.262+0000 7fc319d2d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:49.273 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:49.262+0000 7fc319d2d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:03:49.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:03:49.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:03:49.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:03:49.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:03:49.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:03:49.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:49.406 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:03:49.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:03:49.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:49.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:03:49.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:50.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:50.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:50.558 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:03:50.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:03:50.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:50.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:03:50.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:50.775 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:50.766+0000 7fc319d2d8c0 -1 Falling back to public interface 2026-02-17T12:03:51.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:51.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:51.712 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:03:51.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:03:51.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:51.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:03:51.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:52.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:52.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:52.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:03:52.862 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:03:52.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:52.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:03:53.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:53.781 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:03:53.770+0000 7fc319d2d8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:03:54.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:54.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:54.012 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:03:54.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:03:54.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:54.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:03:54.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:03:55.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:03:55.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:03:55.169 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:03:55.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:03:55.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:03:55.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:03:55.318 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/147522773,v1:127.0.0.1:6811/147522773] [v2:127.0.0.1:6812/147522773,v1:127.0.0.1:6813/147522773] exists,up b15e3214-4568-4115-a9ac-496330818a37 2026-02-17T12:03:55.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:03:55.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:03:55.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:03:55.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5939: TEST_corrupt_snapset_scrub_rep: create_rbd_pool 2026-02-17T12:03:55.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T12:03:55.460 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' does not exist 2026-02-17T12:03:55.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T12:03:55.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:03:55.684 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T12:03:55.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:03:56.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T12:03:57.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5940: TEST_corrupt_snapset_scrub_rep: wait_for_clean 2026-02-17T12:03:57.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:03:57.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:03:57.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:03:57.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:03:57.002 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:03:57.002 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:03:57.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:03:57.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:03:57.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:03:57.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:03:57.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:03:57.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:03:57.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:03:57.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:03:57.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:03:57.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:03:57.276 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:03:57.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:03:57.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:03:57.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:03:57.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T12:03:57.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T12:03:57.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T12:03:57.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:03:57.349 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:03:57.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672962 2026-02-17T12:03:57.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672962 2026-02-17T12:03:57.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-42949672962' 2026-02-17T12:03:57.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:03:57.422 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T12:03:57.422 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:03:57.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:03:57.424 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T12:03:57.424 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:03:57.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T12:03:57.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T12:03:57.426 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836484 2026-02-17T12:03:57.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:03:57.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T12:03:57.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:03:58.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:03:58.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:03:58.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T12:03:58.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:03:58.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672962 2026-02-17T12:03:58.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:03:58.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:03:58.741 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672962 2026-02-17T12:03:58.741 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:03:58.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672962 2026-02-17T12:03:58.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672962' 2026-02-17T12:03:58.743 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672962 2026-02-17T12:03:58.743 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:03:58.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672962 -lt 42949672962 2026-02-17T12:03:58.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:03:58.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:03:58.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:03:59.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T12:03:59.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:03:59.081 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:03:59.082 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:03:59.082 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:03:59.082 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:03:59.082 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:03:59.082 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:03:59.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T12:03:59.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:03:59.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:03:59.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:03:59.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T12:03:59.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:03:59.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:03:59.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5942: TEST_corrupt_snapset_scrub_rep: create_pool foo 1 2026-02-17T12:03:59.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 1 2026-02-17T12:03:59.608 INFO:tasks.workunit.client.0.trial128.stderr:pool 'foo' created 2026-02-17T12:03:59.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:04:00.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5943: TEST_corrupt_snapset_scrub_rep: create_pool csr_pool 1 1 2026-02-17T12:04:00.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create csr_pool 1 1 2026-02-17T12:04:00.792 INFO:tasks.workunit.client.0.trial128.stderr:pool 'csr_pool' created 2026-02-17T12:04:00.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:04:01.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5944: TEST_corrupt_snapset_scrub_rep: wait_for_clean 2026-02-17T12:04:01.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:04:01.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:04:01.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:04:01.804 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:04:01.805 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:04:01.805 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:04:01.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:04:01.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:04:01.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:04:01.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:04:01.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:04:01.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:04:01.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:04:01.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:04:01.932 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:04:02.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:04:02.086 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:04:02.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:04:02.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:04:02.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:04:02.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836486 2026-02-17T12:04:02.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836486 2026-02-17T12:04:02.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486' 2026-02-17T12:04:02.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:04:02.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:04:02.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T12:04:02.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T12:04:02.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836486 1-42949672964' 2026-02-17T12:04:02.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:04:02.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836486 2026-02-17T12:04:02.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:04:02.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:04:02.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836486 2026-02-17T12:04:02.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:04:02.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836486 2026-02-17T12:04:02.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-17T12:04:02.237 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836486 2026-02-17T12:04:02.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:04:02.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-17T12:04:02.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:04:03.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:04:03.404 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:04:03.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836486 2026-02-17T12:04:03.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:04:04.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:04:04.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:04:04.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-17T12:04:04.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:04:04.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T12:04:04.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:04:04.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:04:04.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T12:04:04.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:04:04.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T12:04:04.715 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672964 2026-02-17T12:04:04.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T12:04:04.716 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:04:04.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T12:04:04.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:04:04.884 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:04:04.884 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:04:05.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:04:05.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:04:05.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:04:05.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:04:05.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:04:05.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:04:05.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:04:05.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:04:05.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:04:05.229 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:04:05.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:04:05.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:04:05.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:04:05.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:04:05.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:04:05.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5946: TEST_corrupt_snapset_scrub_rep: seq 1 2 2026-02-17T12:04:05.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5946: TEST_corrupt_snapset_scrub_rep: for i in $(seq 1 $total_objs) 2026-02-17T12:04:05.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5947: TEST_corrupt_snapset_scrub_rep: objname=ROBJ1 2026-02-17T12:04:05.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5948: TEST_corrupt_snapset_scrub_rep: add_something td/osd-scrub-repair csr_pool ROBJ1 2026-02-17T12:04:05.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T12:04:05.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T12:04:05.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ1 2026-02-17T12:04:05.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T12:04:05.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T12:04:05.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T12:04:05.614 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:04:05.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T12:04:05.820 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:04:05.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T12:04:05.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T12:04:05.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:04:05.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5950: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool setomapheader ROBJ1 hdr-ROBJ1 2026-02-17T12:04:05.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5951: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool setomapval ROBJ1 key-ROBJ1 val-ROBJ1 2026-02-17T12:04:05.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5946: TEST_corrupt_snapset_scrub_rep: for i in $(seq 1 $total_objs) 2026-02-17T12:04:05.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5947: TEST_corrupt_snapset_scrub_rep: objname=ROBJ2 2026-02-17T12:04:05.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5948: TEST_corrupt_snapset_scrub_rep: add_something td/osd-scrub-repair csr_pool ROBJ2 2026-02-17T12:04:05.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T12:04:05.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=csr_pool 2026-02-17T12:04:05.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=ROBJ2 2026-02-17T12:04:05.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T12:04:05.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T12:04:05.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T12:04:06.124 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:04:06.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T12:04:06.327 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:04:06.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T12:04:06.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T12:04:06.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool csr_pool put ROBJ2 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:04:06.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5950: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool setomapheader ROBJ2 hdr-ROBJ2 2026-02-17T12:04:06.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5951: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool setomapval ROBJ2 key-ROBJ2 val-ROBJ2 2026-02-17T12:04:06.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5954: TEST_corrupt_snapset_scrub_rep: get_pg csr_pool ROBJ0 2026-02-17T12:04:06.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=csr_pool 2026-02-17T12:04:06.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=ROBJ0 2026-02-17T12:04:06.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map csr_pool ROBJ0 2026-02-17T12:04:06.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T12:04:06.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5954: TEST_corrupt_snapset_scrub_rep: local pg=3.0 2026-02-17T12:04:06.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5955: TEST_corrupt_snapset_scrub_rep: get_primary csr_pool ROBJ0 2026-02-17T12:04:06.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=csr_pool 2026-02-17T12:04:06.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=ROBJ0 2026-02-17T12:04:06.566 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map csr_pool ROBJ0 2026-02-17T12:04:06.566 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:04:06.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5955: TEST_corrupt_snapset_scrub_rep: local primary=1 2026-02-17T12:04:06.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5957: TEST_corrupt_snapset_scrub_rep: rados -p csr_pool mksnap snap1 2026-02-17T12:04:06.809 INFO:tasks.workunit.client.0.trial128.stdout:created pool csr_pool snap snap1 2026-02-17T12:04:06.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5958: TEST_corrupt_snapset_scrub_rep: echo -n head_of_snapshot_data 2026-02-17T12:04:06.812 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5960: TEST_corrupt_snapset_scrub_rep: seq 1 2 2026-02-17T12:04:06.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5960: TEST_corrupt_snapset_scrub_rep: for i in $(seq 1 $total_objs) 2026-02-17T12:04:06.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5961: TEST_corrupt_snapset_scrub_rep: objname=ROBJ1 2026-02-17T12:04:06.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5964: TEST_corrupt_snapset_scrub_rep: expr 1 % 2 2026-02-17T12:04:06.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5964: TEST_corrupt_snapset_scrub_rep: local osd=1 2026-02-17T12:04:06.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5966: TEST_corrupt_snapset_scrub_rep: case $i in 2026-02-17T12:04:06.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5968: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool put ROBJ1 td/osd-scrub-repair/change 2026-02-17T12:04:06.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5969: TEST_corrupt_snapset_scrub_rep: objectstore_tool td/osd-scrub-repair 1 --head ROBJ1 clear-snapset corrupt 2026-02-17T12:04:06.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:04:06.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:04:06.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:04:06.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:04:06.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 1 --head ROBJ1 clear-snapset corrupt 2026-02-17T12:04:06.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:04:06.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:04:06.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:04:06.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:04:06.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:04:06.838 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:04:06.838 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:04:06.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:04:06.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:04:06.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:04:08.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:04:08.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 1 --head ROBJ1 clear-snapset corrupt 2026-02-17T12:04:08.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:04:08.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:04:08.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:04:08.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:04:08.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:04:08.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 --head ROBJ1 clear-snapset corrupt 2026-02-17T12:04:09.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 1 2026-02-17T12:04:09.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:04:09.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:04:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:04:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:04:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:04:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:04:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:04:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:04:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:04:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:04:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:04:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:04:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:04:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:04:09.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:04:09.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:09.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:09.320 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:09.321 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:09.321 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:04:09.321 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:04:09.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:04:09.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:04:09.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:04:09.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:04:09.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:04:09.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:04:09.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:04:09.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:04:09.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:04:09.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:04:09.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:04:09.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:04:09.323 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:04:09.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:04:09.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:04:09.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:04:09.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:04:09.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:04:09.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:04:09.346 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:09.338+0000 7ff30309c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:09.347 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:09.338+0000 7ff30309c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:09.348 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:09.338+0000 7ff30309c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:09.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:04:09.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:04:09.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:04:09.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:04:09.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:04:09.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:09.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:04:09.484 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:04:09.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:09.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:09.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:10.623 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:10.614+0000 7ff30309c8c0 -1 Falling back to public interface 2026-02-17T12:04:10.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:10.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:10.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:04:10.634 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:04:10.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:10.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:10.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:11.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:11.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:11.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:04:11.787 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:04:11.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:11.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:11.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:12.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:12.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:12.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:04:12.936 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:04:12.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:12.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:13.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:13.606 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:13.598+0000 7ff30309c8c0 -1 osd.1 29 log_to_monitors true 2026-02-17T12:04:14.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:14.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:14.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:04:14.092 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:04:14.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:14.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:14.241 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 33 up_thru 33 down_at 30 last_clean_interval [10,29) [v2:127.0.0.1:6810/886444786,v1:127.0.0.1:6811/886444786] [v2:127.0.0.1:6812/886444786,v1:127.0.0.1:6813/886444786] exists,up b15e3214-4568-4115-a9ac-496330818a37 2026-02-17T12:04:14.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:04:14.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:04:14.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:04:14.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:04:14.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:04:14.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:04:14.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:04:14.242 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:04:14.243 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:04:14.243 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:04:14.243 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:04:14.243 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:04:14.243 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:04:14.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:04:14.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:04:14.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:04:14.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:04:14.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:04:14.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:04:14.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:04:14.519 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:04:14.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:04:14.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:04:14.520 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:04:14.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-17T12:04:14.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-17T12:04:14.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-17T12:04:14.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:04:14.592 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:04:14.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=141733920770 2026-02-17T12:04:14.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 141733920770 2026-02-17T12:04:14.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-141733920770' 2026-02-17T12:04:14.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:04:14.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-17T12:04:14.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:04:14.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:04:14.667 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-17T12:04:14.667 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:04:14.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-17T12:04:14.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-17T12:04:14.669 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836489 2026-02-17T12:04:14.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:04:14.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-17T12:04:14.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:04:15.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:04:15.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:04:15.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-17T12:04:15.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:04:16.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:04:16.991 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:04:17.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-17T12:04:17.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:04:17.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-141733920770 2026-02-17T12:04:17.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:04:17.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:04:17.144 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-141733920770 2026-02-17T12:04:17.145 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:04:17.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=141733920770 2026-02-17T12:04:17.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 141733920770' 2026-02-17T12:04:17.146 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 141733920770 2026-02-17T12:04:17.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:04:17.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920770 -lt 141733920770 2026-02-17T12:04:17.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:04:17.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:04:17.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:04:17.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:04:17.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:04:17.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:04:17.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:04:17.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:04:17.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:04:17.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:04:17.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:04:17.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:04:17.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:04:17.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:04:17.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:04:17.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:04:17.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:04:17.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:04:17.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5960: TEST_corrupt_snapset_scrub_rep: for i in $(seq 1 $total_objs) 2026-02-17T12:04:17.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5961: TEST_corrupt_snapset_scrub_rep: objname=ROBJ2 2026-02-17T12:04:17.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5964: TEST_corrupt_snapset_scrub_rep: expr 2 % 2 2026-02-17T12:04:17.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5964: TEST_corrupt_snapset_scrub_rep: local osd=0 2026-02-17T12:04:17.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5966: TEST_corrupt_snapset_scrub_rep: case $i in 2026-02-17T12:04:17.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5973: TEST_corrupt_snapset_scrub_rep: rados --pool csr_pool put ROBJ2 td/osd-scrub-repair/change 2026-02-17T12:04:17.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5974: TEST_corrupt_snapset_scrub_rep: objectstore_tool td/osd-scrub-repair 0 --head ROBJ2 clear-snapset corrupt 2026-02-17T12:04:17.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:04:17.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:04:17.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T12:04:17.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:04:17.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 --head ROBJ2 clear-snapset corrupt 2026-02-17T12:04:17.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:04:17.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:04:17.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T12:04:17.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:04:17.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T12:04:17.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:04:17.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:04:17.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:04:17.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:04:17.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:04:19.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:04:19.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 --head ROBJ2 clear-snapset corrupt 2026-02-17T12:04:19.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:04:19.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:04:19.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:04:19.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:04:19.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:04:19.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 --head ROBJ2 clear-snapset corrupt 2026-02-17T12:04:20.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T12:04:20.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:04:20.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:04:20.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:04:20.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:04:20.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:04:20.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:04:20.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:04:20.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:04:20.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:04:20.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:04:20.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:04:20.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:04:20.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:04:20.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:04:20.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:04:20.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:20.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:20.368 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:20.368 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:20.368 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:04:20.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:04:20.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:04:20.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:04:20.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:04:20.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:04:20.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:04:20.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:04:20.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:04:20.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:04:20.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:04:20.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:04:20.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:04:20.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:04:20.369 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T12:04:20.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:04:20.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T12:04:20.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:04:20.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:04:20.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:04:20.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:04:20.389 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:20.382+0000 7fafa223d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:20.391 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:20.382+0000 7fafa223d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:20.392 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:20.382+0000 7fafa223d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:20.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:04:20.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:04:20.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:04:20.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:04:20.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:04:20.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:20.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:04:20.526 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:04:20.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:20.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:20.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:21.667 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:21.658+0000 7fafa223d8c0 -1 Falling back to public interface 2026-02-17T12:04:21.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:21.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:21.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:04:21.679 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:04:21.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:21.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:21.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:22.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:22.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:22.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:04:22.827 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:04:22.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:22.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:22.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:23.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:23.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:23.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:04:23.980 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:04:23.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:23.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:24.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:24.655 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:24.646+0000 7fafa223d8c0 -1 osd.0 35 log_to_monitors true 2026-02-17T12:04:25.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:25.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:25.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:04:25.132 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:04:25.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:25.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:25.282 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [5,35) [v2:127.0.0.1:6802/980527437,v1:127.0.0.1:6803/980527437] [v2:127.0.0.1:6804/980527437,v1:127.0.0.1:6805/980527437] exists,up b6906687-b163-46f4-a29c-daa47d5e9644 2026-02-17T12:04:25.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:04:25.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:04:25.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:04:25.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:04:25.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:04:25.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:04:25.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:04:25.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:04:25.283 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:04:25.284 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:04:25.284 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:04:25.284 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:04:25.284 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:04:25.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:04:25.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:04:25.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:04:25.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:04:25.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:04:25.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:04:25.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:04:25.561 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:04:25.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:04:25.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:04:25.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:04:25.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724546 2026-02-17T12:04:25.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724546 2026-02-17T12:04:25.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724546' 2026-02-17T12:04:25.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:04:25.632 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:04:25.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=141733920773 2026-02-17T12:04:25.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 141733920773 2026-02-17T12:04:25.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724546 1-141733920773' 2026-02-17T12:04:25.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:04:25.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-167503724546 2026-02-17T12:04:25.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:04:25.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:04:25.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-167503724546 2026-02-17T12:04:25.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:04:25.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724546 2026-02-17T12:04:25.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 167503724546' 2026-02-17T12:04:25.709 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 167503724546 2026-02-17T12:04:25.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:04:25.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 167503724546 2026-02-17T12:04:25.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:04:26.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:04:26.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:04:27.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724546 -lt 167503724546 2026-02-17T12:04:27.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:04:27.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-141733920773 2026-02-17T12:04:27.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:04:27.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:04:27.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-141733920773 2026-02-17T12:04:27.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:04:27.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=141733920773 2026-02-17T12:04:27.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 141733920773' 2026-02-17T12:04:27.028 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 141733920773 2026-02-17T12:04:27.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:04:27.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920773 -lt 141733920773 2026-02-17T12:04:27.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:04:27.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:04:27.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:04:27.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:04:27.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:04:27.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:04:27.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:04:27.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:04:27.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:04:27.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:04:27.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:04:27.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:04:27.533 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:04:27.534 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:04:27.534 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:04:27.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:04:27.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:04:27.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:04:27.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5979: TEST_corrupt_snapset_scrub_rep: rm td/osd-scrub-repair/change 2026-02-17T12:04:27.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5981: TEST_corrupt_snapset_scrub_rep: pg_scrub 3.0 2026-02-17T12:04:27.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=3.0 2026-02-17T12:04:27.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 3.0 2026-02-17T12:04:27.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=3.0 2026-02-17T12:04:27.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T12:04:27.724 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:04:27.724 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:04:27.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:04:27.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:04:27.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:04:28.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T12:04:28.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T12:04:28.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T12:04:28.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T12:04:28.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:04:28.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:04:28.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:04:28.187 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:04:28.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:04:28.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:04:28.188 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:04:28.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724547 2026-02-17T12:04:28.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724547 2026-02-17T12:04:28.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724547' 2026-02-17T12:04:28.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:04:28.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:04:28.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=141733920774 2026-02-17T12:04:28.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 141733920774 2026-02-17T12:04:28.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-167503724547 1-141733920774' 2026-02-17T12:04:28.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:04:28.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-167503724547 2026-02-17T12:04:28.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:04:28.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:04:28.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-167503724547 2026-02-17T12:04:28.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:04:28.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724547 2026-02-17T12:04:28.337 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 167503724547 2026-02-17T12:04:28.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 167503724547' 2026-02-17T12:04:28.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:04:28.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724546 -lt 167503724547 2026-02-17T12:04:28.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:04:29.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:04:29.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:04:29.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724546 -lt 167503724547 2026-02-17T12:04:29.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:04:30.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:04:30.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:04:30.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724548 -lt 167503724547 2026-02-17T12:04:30.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:04:30.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-141733920774 2026-02-17T12:04:30.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:04:30.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:04:30.813 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-141733920774 2026-02-17T12:04:30.813 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:04:30.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=141733920774 2026-02-17T12:04:30.815 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 141733920774 2026-02-17T12:04:30.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 141733920774' 2026-02-17T12:04:30.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:04:30.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920775 -lt 141733920774 2026-02-17T12:04:30.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T12:04:30.978 INFO:tasks.workunit.client.0.trial128.stdout:#---------- 3.0 loop 0 2026-02-17T12:04:30.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 3.0 loop 0' 2026-02-17T12:04:30.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 3.0 2026-02-17T12:04:30.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=3.0 2026-02-17T12:04:30.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T12:04:30.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 3.0 query 2026-02-17T12:04:30.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T12:04:31.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T12:04:31.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T12:04:31.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T12:04:31.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T12:04:31.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 3.0 2026-02-17T12:04:31.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:04:31.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:04:31.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:04:31.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:04:31.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-17T12:04:00.785914+0000 2026-02-17T12:04:31.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 3.0 2026-02-17T12:04:31.341 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 3.0 on osd.1 to scrub 2026-02-17T12:04:31.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 3.0 2026-02-17T12:04:00.785914+0000 2026-02-17T12:04:31.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=3.0 2026-02-17T12:04:31.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:04:00.785914+0000 2026-02-17T12:04:31.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:04:31.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:04:31.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:04:31.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T12:04:31.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:04:31.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:04:31.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:04:31.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:04:31.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:04:00.785914+0000 '>' 2026-02-17T12:04:00.785914+0000 2026-02-17T12:04:31.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:04:32.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:04:32.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:04:32.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T12:04:32.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:04:32.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:04:32.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:04:32.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:04:32.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:04:00.785914+0000 '>' 2026-02-17T12:04:00.785914+0000 2026-02-17T12:04:32.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:04:33.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:04:33.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:04:33.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T12:04:33.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:04:33.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:04:33.658 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:04:33.658 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:04:33.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:04:00.785914+0000 '>' 2026-02-17T12:04:00.785914+0000 2026-02-17T12:04:33.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:04:34.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:04:34.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:04:34.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 3.0 last_scrub_stamp 2026-02-17T12:04:34.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=3.0 2026-02-17T12:04:34.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:04:34.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="3.0") | .last_scrub_stamp' 2026-02-17T12:04:34.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:04:34.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:04:31.720373+0000 '>' 2026-02-17T12:04:00.785914+0000 2026-02-17T12:04:34.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:04:34.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5983: TEST_corrupt_snapset_scrub_rep: rados list-inconsistent-pg csr_pool 2026-02-17T12:04:34.974 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5985: TEST_corrupt_snapset_scrub_rep: jq '. | length' td/osd-scrub-repair/json 2026-02-17T12:04:34.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5985: TEST_corrupt_snapset_scrub_rep: test 1 = 1 2026-02-17T12:04:34.987 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5987: TEST_corrupt_snapset_scrub_rep: jq -r '.[0]' td/osd-scrub-repair/json 2026-02-17T12:04:34.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5987: TEST_corrupt_snapset_scrub_rep: test 3.0 = 3.0 2026-02-17T12:04:34.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5989: TEST_corrupt_snapset_scrub_rep: rados list-inconsistent-obj 3.0 2026-02-17T12:04:35.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5991: TEST_corrupt_snapset_scrub_rep: jq 'def walk(f): 2026-02-17T12:04:35.015 INFO:tasks.workunit.client.0.trial128.stderr: . as $in 2026-02-17T12:04:35.015 INFO:tasks.workunit.client.0.trial128.stderr: | if type == "object" then 2026-02-17T12:04:35.015 INFO:tasks.workunit.client.0.trial128.stderr: reduce keys[] as $key 2026-02-17T12:04:35.015 INFO:tasks.workunit.client.0.trial128.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T12:04:35.016 INFO:tasks.workunit.client.0.trial128.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T12:04:35.016 INFO:tasks.workunit.client.0.trial128.stderr: else f 2026-02-17T12:04:35.016 INFO:tasks.workunit.client.0.trial128.stderr: end; 2026-02-17T12:04:35.016 INFO:tasks.workunit.client.0.trial128.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T12:04:35.016 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T12:04:35.016 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T12:04:35.016 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T12:04:35.016 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.prior_version) else . end)' 2026-02-17T12:04:35.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5991: TEST_corrupt_snapset_scrub_rep: jq .inconsistents 2026-02-17T12:04:35.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5991: TEST_corrupt_snapset_scrub_rep: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T12:04:35.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6157: TEST_corrupt_snapset_scrub_rep: jq 'def walk(f): 2026-02-17T12:04:35.033 INFO:tasks.workunit.client.0.trial128.stderr: . as $in 2026-02-17T12:04:35.033 INFO:tasks.workunit.client.0.trial128.stderr: | if type == "object" then 2026-02-17T12:04:35.033 INFO:tasks.workunit.client.0.trial128.stderr: reduce keys[] as $key 2026-02-17T12:04:35.033 INFO:tasks.workunit.client.0.trial128.stderr: ( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f 2026-02-17T12:04:35.033 INFO:tasks.workunit.client.0.trial128.stderr: elif type == "array" then map( walk(f) ) | f 2026-02-17T12:04:35.033 INFO:tasks.workunit.client.0.trial128.stderr: else f 2026-02-17T12:04:35.033 INFO:tasks.workunit.client.0.trial128.stderr: end; 2026-02-17T12:04:35.033 INFO:tasks.workunit.client.0.trial128.stderr:walk(if type == "object" then del(.mtime) else . end) 2026-02-17T12:04:35.033 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.local_mtime) else . end) 2026-02-17T12:04:35.034 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.last_reqid) else . end) 2026-02-17T12:04:35.034 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.version) else . end) 2026-02-17T12:04:35.034 INFO:tasks.workunit.client.0.trial128.stderr:| walk(if type == "object" then del(.prior_version) else . end)' td/osd-scrub-repair/json 2026-02-17T12:04:35.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6157: TEST_corrupt_snapset_scrub_rep: jq .inconsistents 2026-02-17T12:04:35.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6157: TEST_corrupt_snapset_scrub_rep: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print(json.dumps(ud, sort_keys=True, indent=2))' 2026-02-17T12:04:35.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6158: TEST_corrupt_snapset_scrub_rep: multidiff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T12:04:35.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-repair/checkcsjson td/osd-scrub-repair/csjson 2026-02-17T12:04:35.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6159: TEST_corrupt_snapset_scrub_rep: test no = yes 2026-02-17T12:04:35.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6164: TEST_corrupt_snapset_scrub_rep: test '' = yes 2026-02-17T12:04:35.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6169: TEST_corrupt_snapset_scrub_rep: ERRORS=0 2026-02-17T12:04:35.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6170: TEST_corrupt_snapset_scrub_rep: declare -a err_strings 2026-02-17T12:04:35.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6171: TEST_corrupt_snapset_scrub_rep: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid [0-9]*:.*:::ROBJ1:head : snapset inconsistent' 2026-02-17T12:04:35.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6172: TEST_corrupt_snapset_scrub_rep: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid [0-9]*:.*:::ROBJ2:head : snapset inconsistent' 2026-02-17T12:04:35.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6173: TEST_corrupt_snapset_scrub_rep: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 [0-9]*:.*:::ROBJ1:1 : is an unexpected clone' 2026-02-17T12:04:35.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6174: TEST_corrupt_snapset_scrub_rep: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub : stat mismatch, got 3/4 objects, 1/2 clones, 3/4 dirty, 3/4 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 49/56 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' 2026-02-17T12:04:35.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6175: TEST_corrupt_snapset_scrub_rep: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 0 missing, 2 inconsistent objects' 2026-02-17T12:04:35.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6176: TEST_corrupt_snapset_scrub_rep: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 4 errors' 2026-02-17T12:04:35.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6178: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-17T12:04:35.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6180: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid [0-9]*:.*:::ROBJ1:head : snapset inconsistent' td/osd-scrub-repair/osd.1.log 2026-02-17T12:04:35.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6178: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-17T12:04:35.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6180: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid [0-9]*:.*:::ROBJ2:head : snapset inconsistent' td/osd-scrub-repair/osd.1.log 2026-02-17T12:04:35.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6178: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-17T12:04:35.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6180: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 [0-9]*:.*:::ROBJ1:1 : is an unexpected clone' td/osd-scrub-repair/osd.1.log 2026-02-17T12:04:35.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6178: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-17T12:04:35.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6180: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub : stat mismatch, got 3/4 objects, 1/2 clones, 3/4 dirty, 3/4 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 49/56 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes.' td/osd-scrub-repair/osd.1.log 2026-02-17T12:04:35.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6178: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-17T12:04:35.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6180: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 0 missing, 2 inconsistent objects' td/osd-scrub-repair/osd.1.log 2026-02-17T12:04:35.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6178: TEST_corrupt_snapset_scrub_rep: for err_string in "${err_strings[@]}" 2026-02-17T12:04:35.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6180: TEST_corrupt_snapset_scrub_rep: grep -q 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 4 errors' td/osd-scrub-repair/osd.1.log 2026-02-17T12:04:35.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6187: TEST_corrupt_snapset_scrub_rep: '[' 0 '!=' 0 ']' 2026-02-17T12:04:35.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6193: TEST_corrupt_snapset_scrub_rep: ceph osd pool rm csr_pool csr_pool --yes-i-really-really-mean-it 2026-02-17T12:04:35.272 INFO:tasks.workunit.client.0.trial128.stderr:pool 'csr_pool' removed 2026-02-17T12:04:35.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T12:04:35.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:04:35.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:04:35.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:04:35.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:04:35.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:04:35.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:04:35.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:04:35.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:04:35.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:04:35.393 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:04:35.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:04:35.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:04:35.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:04:35.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:04:35.397 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:04:35.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:04:35.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:04:35.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:04:35.399 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:04:35.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:04:35.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:04:35.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:04:35.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:04:35.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:35.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:04:35.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:04:35.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:04:35.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:04:35.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T12:04:35.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T12:04:35.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T12:04:35.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T12:04:35.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:04:35.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:04:35.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:04:35.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:04:35.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:04:35.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:04:35.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:04:35.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:04:35.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:04:35.421 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:04:35.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:04:35.422 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:04:35.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:04:35.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:04:35.425 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:04:35.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:04:35.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:04:35.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:04:35.427 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:04:35.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:04:35.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:04:35.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:04:35.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:04:35.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:35.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:04:35.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:04:35.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:04:35.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:04:35.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T12:04:35.434 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:04:35.434 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:35.434 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:04:35.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T12:04:35.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:04:35.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:04:35.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:04:35.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T12:04:35.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_list_missing_erasure_coded_appends td/osd-scrub-repair 2026-02-17T12:04:35.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:978: TEST_list_missing_erasure_coded_appends: list_missing_erasure_coded td/osd-scrub-repair false 2026-02-17T12:04:35.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:916: list_missing_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T12:04:35.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:917: list_missing_erasure_coded: local allow_overwrites=false 2026-02-17T12:04:35.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:918: list_missing_erasure_coded: local poolname=ecpool 2026-02-17T12:04:35.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:920: list_missing_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-17T12:04:35.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T12:04:35.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:04:35.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:04:35.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:04:35.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T12:04:35.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T12:04:35.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:04:35.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:35.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:35.459 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:35.459 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:35.459 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:04:35.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:04:35.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:04:35.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:04:35.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:04:35.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:04:35.492 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:04:35.492 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:04:35.492 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:04:35.493 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:04:35.493 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:04:35.493 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:04:35.493 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:04:35.493 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:35.493 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:04:35.494 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:04:35.494 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:04:35.494 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T12:04:35.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:04:35.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:04:35.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:04:35.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:04:35.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:04:35.550 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:04:35.551 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:04:35.551 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:04:35.551 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:04:35.551 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:35.551 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:04:35.551 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:04:35.552 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:04:35.552 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T12:04:35.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:921: list_missing_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-17T12:04:35.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T12:04:35.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:04:35.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:04:35.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:04:35.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T12:04:35.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:04:35.705 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:04:35.705 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:35.705 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:35.706 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:35.706 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:35.706 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:04:35.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:04:35.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:04:35.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:04:35.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: list_missing_erasure_coded: seq 0 2 2026-02-17T12:04:35.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:04:35.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:923: list_missing_erasure_coded: run_osd td/osd-scrub-repair 0 2026-02-17T12:04:35.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:04:35.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:04:35.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:04:35.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:04:35.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:04:35.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:04:35.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:04:35.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:04:35.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:04:35.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:04:35.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:04:35.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:04:35.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:04:35.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:04:35.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:04:35.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:35.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:35.731 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:35.731 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:35.731 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:04:35.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:04:35.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:04:35.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:04:35.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:04:35.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:04:35.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:04:35.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:04:35.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:04:35.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:04:35.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:04:35.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:04:35.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:04:35.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:04:35.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:04:35.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:04:35.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=c2c31842-abfb-45ab-8f00-47cf013d027a 2026-02-17T12:04:35.733 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 c2c31842-abfb-45ab-8f00-47cf013d027a 2026-02-17T12:04:35.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 c2c31842-abfb-45ab-8f00-47cf013d027a' 2026-02-17T12:04:35.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:04:35.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBTWZRpZx4ILBAA/8IM8YcDvmsMVHeO/ozJJA== 2026-02-17T12:04:35.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBTWZRpZx4ILBAA/8IM8YcDvmsMVHeO/ozJJA=="}' 2026-02-17T12:04:35.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new c2c31842-abfb-45ab-8f00-47cf013d027a -i td/osd-scrub-repair/0/new.json 2026-02-17T12:04:35.836 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:04:35.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T12:04:35.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBTWZRpZx4ILBAA/8IM8YcDvmsMVHeO/ozJJA== --osd-uuid c2c31842-abfb-45ab-8f00-47cf013d027a 2026-02-17T12:04:35.860 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:35.853+0000 7fb58024b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:35.861 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:35.857+0000 7fb58024b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:35.862 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:35.857+0000 7fb58024b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:35.863 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:35.857+0000 7fb58024b8c0 -1 bdev(0x5638dd65bc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:04:35.863 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:35.857+0000 7fb58024b8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T12:04:38.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T12:04:38.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:04:38.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:04:38.488 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:04:38.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:04:38.685 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:04:38.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:04:38.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:04:38.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:04:38.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:04:38.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:04:38.709 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:38.701+0000 7f3cac6548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:38.711 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:38.705+0000 7f3cac6548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:38.711 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:38.705+0000 7f3cac6548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:38.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:04:38.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:04:38.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:04:38.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:04:38.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:04:38.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:38.836 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:04:38.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:04:38.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:38.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:38.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:39.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:39.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:39.998 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:04:39.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:04:39.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:39.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:40.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:40.191 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:40.185+0000 7f3cac6548c0 -1 Falling back to public interface 2026-02-17T12:04:41.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:41.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:41.156 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:04:41.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:04:41.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:41.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:41.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:42.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:42.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:42.313 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:04:42.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:04:42.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:42.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:42.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:43.231 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:43.225+0000 7f3cac6548c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:04:43.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:43.469 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:04:43.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:04:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:43.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:44.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:44.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:44.627 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:04:44.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:04:44.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:44.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:04:44.783 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/331243386,v1:127.0.0.1:6803/331243386] [v2:127.0.0.1:6804/331243386,v1:127.0.0.1:6805/331243386] exists,up c2c31842-abfb-45ab-8f00-47cf013d027a 2026-02-17T12:04:44.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:04:44.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:04:44.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:04:44.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:04:44.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:923: list_missing_erasure_coded: run_osd td/osd-scrub-repair 1 2026-02-17T12:04:44.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:04:44.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:04:44.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:04:44.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:04:44.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:04:44.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:04:44.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:04:44.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:04:44.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:04:44.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:04:44.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:04:44.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:04:44.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:04:44.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:04:44.790 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:04:44.790 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:44.790 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:44.790 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:44.791 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:44.791 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:04:44.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:04:44.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:04:44.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:04:44.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:04:44.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:04:44.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:04:44.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:04:44.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:04:44.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:04:44.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:04:44.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:04:44.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:04:44.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:04:44.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:04:44.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:04:44.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=7ea262d3-dac5-4546-8d9d-bea424f0e203 2026-02-17T12:04:44.795 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 7ea262d3-dac5-4546-8d9d-bea424f0e203 2026-02-17T12:04:44.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 7ea262d3-dac5-4546-8d9d-bea424f0e203' 2026-02-17T12:04:44.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:04:44.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBcWZRpo214LxAAeTq1jO96JFEcx4i0ZBZwgg== 2026-02-17T12:04:44.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBcWZRpo214LxAAeTq1jO96JFEcx4i0ZBZwgg=="}' 2026-02-17T12:04:44.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 7ea262d3-dac5-4546-8d9d-bea424f0e203 -i td/osd-scrub-repair/1/new.json 2026-02-17T12:04:44.951 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:04:44.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T12:04:44.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBcWZRpo214LxAAeTq1jO96JFEcx4i0ZBZwgg== --osd-uuid 7ea262d3-dac5-4546-8d9d-bea424f0e203 2026-02-17T12:04:44.978 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:44.973+0000 7f229f3f98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:44.980 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:44.973+0000 7f229f3f98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:44.982 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:44.973+0000 7f229f3f98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:44.982 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:44.977+0000 7f229f3f98c0 -1 bdev(0x55aadcbb3c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:04:44.983 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:44.977+0000 7f229f3f98c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T12:04:47.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T12:04:47.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:04:47.252 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:04:47.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:04:47.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:04:47.451 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:04:47.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:04:47.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:04:47.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:04:47.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:04:47.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:04:47.473 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:47.465+0000 7fd5835378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:47.474 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:47.469+0000 7fd5835378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:47.475 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:47.469+0000 7fd5835378c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:47.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:04:47.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:04:47.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:04:47.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:04:47.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:04:47.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:47.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:04:47.602 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:04:47.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:47.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:47.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:48.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:48.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:48.756 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:04:48.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:04:48.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:48.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:48.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:48.971 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:48.965+0000 7fd5835378c0 -1 Falling back to public interface 2026-02-17T12:04:49.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:49.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:49.916 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:04:49.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:04:49.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:49.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:50.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:51.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:51.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:51.076 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:04:51.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:04:51.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:51.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:51.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:51.953 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:51.945+0000 7fd5835378c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:04:52.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:52.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:52.232 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:04:52.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:04:52.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:52.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:52.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:53.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:53.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:53.394 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:04:53.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:04:53.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:53.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:04:53.545 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1882896795,v1:127.0.0.1:6811/1882896795] [v2:127.0.0.1:6812/1882896795,v1:127.0.0.1:6813/1882896795] exists,up 7ea262d3-dac5-4546-8d9d-bea424f0e203 2026-02-17T12:04:53.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:04:53.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:04:53.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:04:53.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:04:53.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:923: list_missing_erasure_coded: run_osd td/osd-scrub-repair 2 2026-02-17T12:04:53.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:04:53.547 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:04:53.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:04:53.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:04:53.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:04:53.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:04:53.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:04:53.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:04:53.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:04:53.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:04:53.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:04:53.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:04:53.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:04:53.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:04:53.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T12:04:53.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:04:53.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=1ab819a3-6f70-4bd2-8b74-48626c1068d4 2026-02-17T12:04:53.551 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 1ab819a3-6f70-4bd2-8b74-48626c1068d4 2026-02-17T12:04:53.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 1ab819a3-6f70-4bd2-8b74-48626c1068d4' 2026-02-17T12:04:53.551 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:04:53.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBlWZRpU5BRIRAAGhAnfTfRRgbvbkG7R1S2tQ== 2026-02-17T12:04:53.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBlWZRpU5BRIRAAGhAnfTfRRgbvbkG7R1S2tQ=="}' 2026-02-17T12:04:53.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 1ab819a3-6f70-4bd2-8b74-48626c1068d4 -i td/osd-scrub-repair/2/new.json 2026-02-17T12:04:53.722 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:04:53.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T12:04:53.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBlWZRpU5BRIRAAGhAnfTfRRgbvbkG7R1S2tQ== --osd-uuid 1ab819a3-6f70-4bd2-8b74-48626c1068d4 2026-02-17T12:04:53.749 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:53.745+0000 7f30de3128c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:53.751 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:53.745+0000 7f30de3128c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:53.752 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:53.745+0000 7f30de3128c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:53.752 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:53.745+0000 7f30de3128c0 -1 bdev(0x55ce262d7c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:04:53.752 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:53.745+0000 7f30de3128c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T12:04:56.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T12:04:56.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:04:56.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:04:56.044 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T12:04:56.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:04:56.243 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T12:04:56.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:04:56.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:04:56.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:04:56.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:04:56.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:04:56.266 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:56.261+0000 7f64573078c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:56.268 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:56.261+0000 7f64573078c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:56.269 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:56.261+0000 7f64573078c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:04:56.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:04:56.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:04:56.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:04:56.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:04:56.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:04:56.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:56.394 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:04:56.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:04:56.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:56.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:04:56.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:57.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:57.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:57.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:04:57.547 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:04:57.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:57.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:04:57.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:57.759 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:04:57.753+0000 7f64573078c0 -1 Falling back to public interface 2026-02-17T12:04:58.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:58.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:58.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:04:58.701 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:04:58.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:58.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:04:58.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:04:59.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:04:59.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:04:59.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:04:59.860 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:04:59.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:04:59.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:05:00.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:00.731 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:00.725+0000 7f64573078c0 -1 osd.2 0 log_to_monitors true 2026-02-17T12:05:01.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:01.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:01.017 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:05:01.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:05:01.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:01.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:05:01.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:02.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:02.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:02.174 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:05:02.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:05:02.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:02.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:05:02.326 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2045658203,v1:127.0.0.1:6819/2045658203] [v2:127.0.0.1:6820/2045658203,v1:127.0.0.1:6821/2045658203] exists,up 1ab819a3-6f70-4bd2-8b74-48626c1068d4 2026-02-17T12:05:02.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:05:02.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:05:02.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:05:02.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:925: list_missing_erasure_coded: create_rbd_pool 2026-02-17T12:05:02.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T12:05:02.470 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' does not exist 2026-02-17T12:05:02.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T12:05:02.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:05:02.696 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T12:05:02.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:05:03.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T12:05:03.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:926: list_missing_erasure_coded: wait_for_clean 2026-02-17T12:05:03.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:05:03.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:05:03.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:05:03.987 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:05:03.988 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:05:03.988 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:05:03.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:05:03.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:05:03.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:05:04.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:05:04.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:05:04.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:05:04.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:05:04.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:05:04.111 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:05:04.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:05:04.265 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:05:04.265 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:05:04.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:05:04.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:04.266 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:05:04.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T12:05:04.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T12:05:04.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T12:05:04.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:04.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:05:04.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T12:05:04.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T12:05:04.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-17T12:05:04.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:04.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:05:04.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509442 2026-02-17T12:05:04.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509442 2026-02-17T12:05:04.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-64424509442' 2026-02-17T12:05:04.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:05:04.487 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T12:05:04.487 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:05:04.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:05:04.489 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T12:05:04.489 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:05:04.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T12:05:04.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T12:05:04.491 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836485 2026-02-17T12:05:04.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:05:04.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T12:05:04.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:05:05.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:05:05.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:05:05.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T12:05:05.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:05:06.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:05:06.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:05:06.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T12:05:06.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:05:06.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T12:05:06.954 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:05:06.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:05:06.956 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T12:05:06.956 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:05:06.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T12:05:06.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T12:05:06.958 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672964 2026-02-17T12:05:06.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:05:07.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T12:05:07.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:05:07.114 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509442 2026-02-17T12:05:07.114 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:05:07.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:05:07.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509442 2026-02-17T12:05:07.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:05:07.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509442 2026-02-17T12:05:07.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509442' 2026-02-17T12:05:07.119 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 64424509442 2026-02-17T12:05:07.119 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:05:07.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509442 -lt 64424509442 2026-02-17T12:05:07.267 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:05:07.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:05:07.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:05:07.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T12:05:07.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:05:07.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:05:07.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:05:07.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:05:07.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:05:07.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:05:07.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:05:07.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T12:05:07.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:05:07.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:05:07.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:05:07.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T12:05:07.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:05:07.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:05:07.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:928: list_missing_erasure_coded: create_ec_pool ecpool false k=2 m=1 2026-02-17T12:05:07.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T12:05:07.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T12:05:07.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=false 2026-02-17T12:05:07.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T12:05:07.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-17T12:05:07.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T12:05:07.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T12:05:08.284 INFO:tasks.workunit.client.0.trial128.stderr:pool 'ecpool' created 2026-02-17T12:05:08.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:05:09.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' false = true ']' 2026-02-17T12:05:09.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T12:05:09.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:05:09.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:05:09.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:05:09.299 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:05:09.299 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:05:09.300 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:05:09.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:05:09.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:05:09.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:05:09.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:05:09.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:05:09.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:05:09.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:05:09.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:05:09.422 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:05:09.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:05:09.574 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:05:09.574 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:05:09.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:05:09.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:09.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:05:09.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836488 2026-02-17T12:05:09.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836488 2026-02-17T12:05:09.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488' 2026-02-17T12:05:09.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:09.649 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:05:09.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672966 2026-02-17T12:05:09.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672966 2026-02-17T12:05:09.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-42949672966' 2026-02-17T12:05:09.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:09.720 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:05:09.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509444 2026-02-17T12:05:09.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509444 2026-02-17T12:05:09.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-42949672966 2-64424509444' 2026-02-17T12:05:09.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:05:09.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836488 2026-02-17T12:05:09.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:05:09.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:05:09.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836488 2026-02-17T12:05:09.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:05:09.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836488 2026-02-17T12:05:09.796 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836488 2026-02-17T12:05:09.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-17T12:05:09.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:05:09.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836488 2026-02-17T12:05:09.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:05:10.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:05:10.959 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:05:11.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-17T12:05:11.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:05:11.114 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672966 2026-02-17T12:05:11.114 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:05:11.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:05:11.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672966 2026-02-17T12:05:11.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:05:11.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672966 2026-02-17T12:05:11.119 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672966 2026-02-17T12:05:11.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2026-02-17T12:05:11.119 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:05:11.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672966 -lt 42949672966 2026-02-17T12:05:11.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:05:11.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509444 2026-02-17T12:05:11.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:05:11.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:05:11.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509444 2026-02-17T12:05:11.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:05:11.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509444 2026-02-17T12:05:11.277 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 64424509444 2026-02-17T12:05:11.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509444' 2026-02-17T12:05:11.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:05:11.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509444 -lt 64424509444 2026-02-17T12:05:11.444 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:05:11.445 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:05:11.445 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:05:11.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:05:11.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:05:11.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:05:11.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:05:11.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:05:11.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:05:11.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:05:11.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:05:11.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:05:11.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:05:11.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:05:11.786 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:05:11.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:05:11.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:05:11.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:05:11.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T12:05:11.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:931: list_missing_erasure_coded: add_something td/osd-scrub-repair ecpool MOBJ0 2026-02-17T12:05:11.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T12:05:11.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T12:05:11.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=MOBJ0 2026-02-17T12:05:11.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T12:05:11.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T12:05:11.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T12:05:12.131 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:05:12.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T12:05:12.331 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:05:12.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T12:05:12.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T12:05:12.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put MOBJ0 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:05:12.367 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:932: list_missing_erasure_coded: get_osds ecpool MOBJ0 2026-02-17T12:05:12.367 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T12:05:12.367 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=MOBJ0 2026-02-17T12:05:12.368 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool MOBJ0 2026-02-17T12:05:12.368 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:05:12.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=2 2026-02-17T12:05:12.517 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:05:12.517 INFO:tasks.workunit.client.0.trial128.stderr:0' 2026-02-17T12:05:12.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 2 1 0 2026-02-17T12:05:12.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:932: list_missing_erasure_coded: osds0=('2' '1' '0') 2026-02-17T12:05:12.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:932: list_missing_erasure_coded: local -a osds0 2026-02-17T12:05:12.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:935: list_missing_erasure_coded: add_something td/osd-scrub-repair ecpool MOBJ1 2026-02-17T12:05:12.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T12:05:12.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T12:05:12.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=MOBJ1 2026-02-17T12:05:12.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T12:05:12.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T12:05:12.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T12:05:12.716 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:05:12.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T12:05:12.921 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:05:12.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T12:05:12.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T12:05:12.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put MOBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:05:12.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:936: list_missing_erasure_coded: get_osds ecpool MOBJ1 2026-02-17T12:05:12.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T12:05:12.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=MOBJ1 2026-02-17T12:05:12.960 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool MOBJ1 2026-02-17T12:05:12.960 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:05:13.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=2 2026-02-17T12:05:13.116 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:05:13.116 INFO:tasks.workunit.client.0.trial128.stderr:0' 2026-02-17T12:05:13.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 2 1 0 2026-02-17T12:05:13.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:936: list_missing_erasure_coded: osds1=('2' '1' '0') 2026-02-17T12:05:13.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:936: list_missing_erasure_coded: local -a osds1 2026-02-17T12:05:13.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: list_missing_erasure_coded: seq 0 2 2026-02-17T12:05:13.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:05:13.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:940: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T12:05:13.118 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:05:13.118 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:05:13.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:05:13.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:05:13.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:05:14.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:05:14.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:05:14.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:940: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:05:14.430 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:05:14.430 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:05:14.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:05:14.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:05:14.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:05:15.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:05:15.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:05:15.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:940: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T12:05:15.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:05:15.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:05:15.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:05:15.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:05:15.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:05:17.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:05:17.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:943: list_missing_erasure_coded: id=2 2026-02-17T12:05:17.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:944: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 MOBJ0 remove 2026-02-17T12:05:17.727 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#2:a9030505:::MOBJ0:head# 2026-02-17T12:05:18.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:946: list_missing_erasure_coded: id=1 2026-02-17T12:05:18.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:947: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 MOBJ0 remove 2026-02-17T12:05:18.907 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#2:a9030505:::MOBJ0:head# 2026-02-17T12:05:19.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:950: list_missing_erasure_coded: id=1 2026-02-17T12:05:19.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:951: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 MOBJ1 remove 2026-02-17T12:05:20.093 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#2:8dd16d55:::MOBJ1:head# 2026-02-17T12:05:20.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:953: list_missing_erasure_coded: id=0 2026-02-17T12:05:20.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:954: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 MOBJ1 remove 2026-02-17T12:05:21.271 INFO:tasks.workunit.client.0.trial128.stdout:remove 2#2:8dd16d55:::MOBJ1:head# 2026-02-17T12:05:21.807 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: list_missing_erasure_coded: seq 0 2 2026-02-17T12:05:21.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:05:21.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:958: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 0 2026-02-17T12:05:21.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:05:21.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:05:21.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:05:21.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:05:21.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:05:21.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:05:21.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:05:21.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:05:21.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:05:21.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:05:21.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:05:21.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:05:21.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:05:21.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:05:21.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:05:21.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:05:21.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:05:21.814 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:05:21.814 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:21.814 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:05:21.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:05:21.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:05:21.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:05:21.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:05:21.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:05:21.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:05:21.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:05:21.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:05:21.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:05:21.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:05:21.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:05:21.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:05:21.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:05:21.816 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T12:05:21.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:05:21.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T12:05:21.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:05:21.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:05:21.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:05:21.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:05:21.835 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:21.829+0000 7fe4acef78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:21.837 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:21.829+0000 7fe4acef78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:21.838 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:21.833+0000 7fe4acef78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:21.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:05:21.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:05:21.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:05:21.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:05:21.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:05:21.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:21.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:05:21.963 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:05:21.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:21.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:05:22.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:23.067 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:23.061+0000 7fe4acef78c0 -1 Falling back to public interface 2026-02-17T12:05:23.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:23.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:23.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:05:23.119 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:05:23.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:23.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:05:23.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:24.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:24.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:24.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:05:24.275 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:05:24.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:24.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:05:24.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:25.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:25.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:25.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:05:25.433 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:05:25.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:25.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:05:25.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:26.050 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:26.045+0000 7fe4acef78c0 -1 osd.0 31 log_to_monitors true 2026-02-17T12:05:26.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:26.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:26.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:05:26.590 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:05:26.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:26.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:05:26.749 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 38 up_thru 38 down_at 32 last_clean_interval [5,31) [v2:127.0.0.1:6802/2935696773,v1:127.0.0.1:6803/2935696773] [v2:127.0.0.1:6804/2935696773,v1:127.0.0.1:6805/2935696773] exists,up c2c31842-abfb-45ab-8f00-47cf013d027a 2026-02-17T12:05:26.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:05:26.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:05:26.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:05:26.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:05:26.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:958: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 1 2026-02-17T12:05:26.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:05:26.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:05:26.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:05:26.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:05:26.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:05:26.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:05:26.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:05:26.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:05:26.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:05:26.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:05:26.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:05:26.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:05:26.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:05:26.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:05:26.751 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:05:26.751 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:05:26.751 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:05:26.751 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:05:26.751 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:26.751 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:05:26.752 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:05:26.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:05:26.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:05:26.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:05:26.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:05:26.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:05:26.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:05:26.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:05:26.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:05:26.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:05:26.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:05:26.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:05:26.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:05:26.754 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:05:26.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:05:26.754 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:05:26.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:05:26.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:05:26.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:05:26.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:05:26.768 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:26.760+0000 7f7aaf2e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:26.769 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:26.764+0000 7f7aaf2e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:26.770 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:26.764+0000 7f7aaf2e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:26.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:05:26.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:05:26.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:05:26.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:05:26.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:05:26.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:26.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:05:26.903 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:05:26.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:26.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:05:27.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:28.039 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:28.032+0000 7f7aaf2e08c0 -1 Falling back to public interface 2026-02-17T12:05:28.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:28.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:28.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:05:28.060 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:05:28.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:28.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:05:28.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:29.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:29.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:29.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:05:29.214 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:05:29.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:29.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:05:29.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:30.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:30.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:30.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:05:30.373 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:05:30.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:30.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:05:30.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:31.013 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:31.004+0000 7f7aaf2e08c0 -1 osd.1 33 log_to_monitors true 2026-02-17T12:05:31.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:31.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:31.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:05:31.530 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:05:31.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:31.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:05:31.686 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 41 up_thru 41 down_at 34 last_clean_interval [10,33) [v2:127.0.0.1:6810/3086593641,v1:127.0.0.1:6811/3086593641] [v2:127.0.0.1:6812/3086593641,v1:127.0.0.1:6813/3086593641] exists,up 7ea262d3-dac5-4546-8d9d-bea424f0e203 2026-02-17T12:05:31.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:05:31.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:05:31.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:05:31.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:05:31.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:958: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 2 2026-02-17T12:05:31.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:05:31.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:05:31.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:05:31.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:05:31.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T12:05:31.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:05:31.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:05:31.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:05:31.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:05:31.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T12:05:31.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T12:05:31.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:05:31.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:05:31.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:05:31.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:05:31.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:05:31.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:05:31.688 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:05:31.688 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:31.688 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:05:31.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:05:31.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:05:31.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:05:31.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:05:31.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:05:31.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:05:31.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:05:31.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:05:31.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:05:31.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:05:31.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:05:31.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T12:05:31.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:05:31.689 INFO:tasks.workunit.client.0.trial128.stderr:start osd.2 2026-02-17T12:05:31.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:05:31.691 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T12:05:31.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:05:31.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:05:31.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:05:31.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:05:31.701 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:31.692+0000 7f291c9a28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:31.702 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:31.696+0000 7f291c9a28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:31.703 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:31.696+0000 7f291c9a28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:31.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:05:31.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:05:31.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:05:31.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:05:31.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:05:31.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:31.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:05:31.838 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:05:31.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:31.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:05:31.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:32.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:32.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:32.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:05:32.997 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:05:32.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:32.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:05:33.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:33.163 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:33.156+0000 7f291c9a28c0 -1 Falling back to public interface 2026-02-17T12:05:34.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:34.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:34.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:05:34.155 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:05:34.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:34.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:05:34.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:35.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:35.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:35.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:05:35.310 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:05:35.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:35.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:05:35.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:36.118 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:36.112+0000 7f291c9a28c0 -1 osd.2 35 log_to_monitors true 2026-02-17T12:05:36.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:36.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:36.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:05:36.473 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:05:36.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:36.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:05:36.631 INFO:tasks.workunit.client.0.trial128.stderr:osd.2 up in weight 1 up_from 45 up_thru 45 down_at 36 last_clean_interval [15,35) [v2:127.0.0.1:6818/1561225180,v1:127.0.0.1:6819/1561225180] [v2:127.0.0.1:6820/1561225180,v1:127.0.0.1:6821/1561225180] exists,up 1ab819a3-6f70-4bd2-8b74-48626c1068d4 2026-02-17T12:05:36.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:05:36.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:05:36.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:05:36.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:960: list_missing_erasure_coded: create_rbd_pool 2026-02-17T12:05:36.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T12:05:36.872 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' removed 2026-02-17T12:05:36.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T12:05:36.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:05:37.076 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T12:05:37.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:05:38.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T12:05:38.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:961: list_missing_erasure_coded: wait_for_clean 2026-02-17T12:05:38.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:05:38.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:05:38.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:05:38.398 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:05:38.398 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:05:38.398 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:05:38.398 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:05:38.399 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:05:38.399 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:05:38.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:05:38.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:05:38.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:05:38.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:05:38.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:05:38.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:05:38.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:05:38.665 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:05:38.666 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:05:38.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:05:38.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:38.666 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:05:38.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757252 2026-02-17T12:05:38.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757252 2026-02-17T12:05:38.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-163208757252' 2026-02-17T12:05:38.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:38.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:05:38.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=176093659139 2026-02-17T12:05:38.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 176093659139 2026-02-17T12:05:38.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-163208757252 1-176093659139' 2026-02-17T12:05:38.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:05:38.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:05:38.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=193273528322 2026-02-17T12:05:38.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 193273528322 2026-02-17T12:05:38.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-163208757252 1-176093659139 2-193273528322' 2026-02-17T12:05:38.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:05:38.877 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-163208757252 2026-02-17T12:05:38.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:05:38.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:05:38.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-163208757252 2026-02-17T12:05:38.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:05:38.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757252 2026-02-17T12:05:38.882 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 163208757252 2026-02-17T12:05:38.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 163208757252' 2026-02-17T12:05:38.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:05:39.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757251 -lt 163208757252 2026-02-17T12:05:39.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:05:40.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:05:40.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:05:40.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757252 -lt 163208757252 2026-02-17T12:05:40.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:05:40.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-176093659139 2026-02-17T12:05:40.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:05:40.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:05:40.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-176093659139 2026-02-17T12:05:40.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:05:40.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=176093659139 2026-02-17T12:05:40.204 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 176093659139 2026-02-17T12:05:40.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 176093659139' 2026-02-17T12:05:40.204 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:05:40.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 176093659139 -lt 176093659139 2026-02-17T12:05:40.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:05:40.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-193273528322 2026-02-17T12:05:40.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:05:40.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:05:40.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-193273528322 2026-02-17T12:05:40.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:05:40.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=193273528322 2026-02-17T12:05:40.372 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 193273528322 2026-02-17T12:05:40.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 193273528322' 2026-02-17T12:05:40.373 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:05:40.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 193273528322 -lt 193273528322 2026-02-17T12:05:40.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:05:40.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:05:40.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:05:40.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:05:40.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:05:40.720 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:05:40.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:05:40.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:05:40.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:05:40.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:05:40.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:05:40.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:05:40.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:05:40.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:05:40.875 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:05:41.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:05:41.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:05:41.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:05:41.054 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:964: list_missing_erasure_coded: get_pg ecpool MOBJ0 2026-02-17T12:05:41.054 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T12:05:41.054 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=MOBJ0 2026-02-17T12:05:41.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T12:05:41.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool MOBJ0 2026-02-17T12:05:41.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:964: list_missing_erasure_coded: local pg=2.0 2026-02-17T12:05:41.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:968: list_missing_erasure_coded: repair 2.0 2026-02-17T12:05:41.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T12:05:41.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T12:05:41.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:05:41.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:05:41.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:05:41.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:05:41.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T12:05:08.279604+0000 2026-02-17T12:05:41.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T12:05:41.499 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.2 to repair 2026-02-17T12:05:41.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T12:05:08.279604+0000 2026-02-17T12:05:41.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T12:05:41.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:05:08.279604+0000 2026-02-17T12:05:41.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:05:41.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:05:41.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:05:41.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:05:41.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:05:41.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:05:41.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:05:41.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:05:41.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:05:08.279604+0000 '>' 2026-02-17T12:05:08.279604+0000 2026-02-17T12:05:41.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:05:42.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:05:42.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:05:42.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:05:42.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:05:42.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:05:42.663 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:05:42.663 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:05:42.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:05:08.279604+0000 '>' 2026-02-17T12:05:08.279604+0000 2026-02-17T12:05:42.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:05:43.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:05:43.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:05:43.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:05:43.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:05:43.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:05:43.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:05:43.818 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:05:43.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:05:08.279604+0000 '>' 2026-02-17T12:05:08.279604+0000 2026-02-17T12:05:43.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:05:44.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:05:44.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:05:44.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:05:44.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:05:44.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:05:44.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:05:44.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:05:45.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:05:08.279604+0000 '>' 2026-02-17T12:05:08.279604+0000 2026-02-17T12:05:45.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:05:46.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:05:46.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:05:46.120 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:05:46.121 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:05:46.121 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:05:46.121 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:05:46.121 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:05:46.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:05:08.279604+0000 '>' 2026-02-17T12:05:08.279604+0000 2026-02-17T12:05:46.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:05:47.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:05:47.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:05:47.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:05:47.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:05:47.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:05:47.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:05:47.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:05:47.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:05:08.279604+0000 '>' 2026-02-17T12:05:08.279604+0000 2026-02-17T12:05:47.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:05:48.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:05:48.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:05:48.425 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:05:48.425 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:05:48.425 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:05:48.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:05:48.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:05:48.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:05:42.076975+0000 '>' 2026-02-17T12:05:08.279604+0000 2026-02-17T12:05:48.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:05:48.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:970: list_missing_erasure_coded: seq 0 120 2026-02-17T12:05:48.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:970: list_missing_erasure_coded: for i in $(seq 0 120) 2026-02-17T12:05:48.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:971: list_missing_erasure_coded: '[' 0 -lt 60 ']' 2026-02-17T12:05:48.588 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:972: list_missing_erasure_coded: ceph pg 2.0 list_unfound 2026-02-17T12:05:48.588 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:972: list_missing_erasure_coded: egrep 'MOBJ0|MOBJ1' 2026-02-17T12:05:48.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:972: list_missing_erasure_coded: wc -l 2026-02-17T12:05:48.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:972: list_missing_erasure_coded: matches=2 2026-02-17T12:05:48.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:973: list_missing_erasure_coded: '[' 2 -eq 2 ']' 2026-02-17T12:05:48.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:973: list_missing_erasure_coded: break 2026-02-17T12:05:48.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T12:05:48.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:05:48.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:05:48.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:05:48.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:05:48.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:05:48.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:05:48.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:05:48.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:05:48.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:05:48.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:05:48.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:05:48.774 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:05:48.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:05:48.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:05:48.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:05:48.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:05:48.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:05:48.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:05:48.778 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:05:48.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:05:48.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:05:48.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:05:48.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:05:48.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:48.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:05:48.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:05:48.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:05:48.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:05:48.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T12:05:48.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T12:05:48.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T12:05:48.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T12:05:48.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:05:48.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:05:48.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:05:48.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:05:48.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:05:48.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:05:48.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:05:48.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:05:48.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:05:48.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:05:48.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:05:48.804 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:05:48.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:05:48.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:05:48.805 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:05:48.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:05:48.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:05:48.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:05:48.807 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:05:48.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:05:48.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:05:48.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:05:48.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:05:48.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:48.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:05:48.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:05:48.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:05:48.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:05:48.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T12:05:48.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:05:48.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:48.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:05:48.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T12:05:48.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:05:48.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:05:48.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:05:48.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T12:05:48.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_list_missing_erasure_coded_overwrites td/osd-scrub-repair 2026-02-17T12:05:48.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:982: TEST_list_missing_erasure_coded_overwrites: '[' true = true ']' 2026-02-17T12:05:48.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:983: TEST_list_missing_erasure_coded_overwrites: list_missing_erasure_coded td/osd-scrub-repair true 2026-02-17T12:05:48.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:916: list_missing_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T12:05:48.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:917: list_missing_erasure_coded: local allow_overwrites=true 2026-02-17T12:05:48.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:918: list_missing_erasure_coded: local poolname=ecpool 2026-02-17T12:05:48.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:920: list_missing_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-17T12:05:48.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T12:05:48.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:05:48.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:05:48.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:05:48.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T12:05:48.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T12:05:48.840 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:05:48.840 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:05:48.840 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:05:48.840 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:05:48.840 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:48.841 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:05:48.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:05:48.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:05:48.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:05:48.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:05:48.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:05:48.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:05:48.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:05:48.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:05:48.871 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:05:48.871 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:05:48.871 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:05:48.872 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:05:48.872 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:48.872 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:05:48.872 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:05:48.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:05:48.873 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T12:05:48.929 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:05:48.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:05:48.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:05:48.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:05:48.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:05:48.931 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:05:48.931 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:05:48.931 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:05:48.931 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:05:48.932 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:48.932 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:05:48.932 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:05:48.932 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:05:48.932 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T12:05:48.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:921: list_missing_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-17T12:05:48.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T12:05:48.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:05:48.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:05:48.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:05:48.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T12:05:48.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:05:49.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:05:49.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:05:49.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:05:49.083 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:05:49.083 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:49.083 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:05:49.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:05:49.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:05:49.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:05:49.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: list_missing_erasure_coded: seq 0 2 2026-02-17T12:05:49.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:05:49.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:923: list_missing_erasure_coded: run_osd td/osd-scrub-repair 0 2026-02-17T12:05:49.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:05:49.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:05:49.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:05:49.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:05:49.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:05:49.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:05:49.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:05:49.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:05:49.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:05:49.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:05:49.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:05:49.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:05:49.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:05:49.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:05:49.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:05:49.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:05:49.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:05:49.108 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:05:49.108 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:49.108 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:05:49.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:05:49.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:05:49.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:05:49.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:05:49.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:05:49.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:05:49.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:05:49.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:05:49.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:05:49.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:05:49.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:05:49.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:05:49.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:05:49.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:05:49.111 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:05:49.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=9544989c-8f90-45d0-9efe-5ab41b5e7f5c 2026-02-17T12:05:49.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 9544989c-8f90-45d0-9efe-5ab41b5e7f5c' 2026-02-17T12:05:49.112 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 9544989c-8f90-45d0-9efe-5ab41b5e7f5c 2026-02-17T12:05:49.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:05:49.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCdWZRpU37iBhAApzVb0/j/m6NOn8krKfmwfA== 2026-02-17T12:05:49.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCdWZRpU37iBhAApzVb0/j/m6NOn8krKfmwfA=="}' 2026-02-17T12:05:49.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 9544989c-8f90-45d0-9efe-5ab41b5e7f5c -i td/osd-scrub-repair/0/new.json 2026-02-17T12:05:49.211 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:05:49.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T12:05:49.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCdWZRpU37iBhAApzVb0/j/m6NOn8krKfmwfA== --osd-uuid 9544989c-8f90-45d0-9efe-5ab41b5e7f5c 2026-02-17T12:05:49.235 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:49.228+0000 7ff7349428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:49.236 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:49.228+0000 7ff7349428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:49.237 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:49.228+0000 7ff7349428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:49.238 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:49.232+0000 7ff7349428c0 -1 bdev(0x559052ac5c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:05:49.238 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:49.232+0000 7ff7349428c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T12:05:51.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T12:05:51.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:05:51.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:05:51.832 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:05:51.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:05:52.033 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:05:52.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:05:52.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:05:52.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:05:52.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:05:52.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:05:52.058 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:52.052+0000 7faa117aa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:52.060 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:52.052+0000 7faa117aa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:52.061 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:52.052+0000 7faa117aa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:52.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:05:52.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:05:52.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:05:52.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:05:52.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:05:52.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:52.185 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:05:52.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:05:52.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:52.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:05:52.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:53.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:53.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:53.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:05:53.344 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:05:53.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:53.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:05:53.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:53.555 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:53.548+0000 7faa117aa8c0 -1 Falling back to public interface 2026-02-17T12:05:54.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:54.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:54.500 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:05:54.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:05:54.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:54.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:05:54.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:55.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:55.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:55.657 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:05:55.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:05:55.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:55.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:05:55.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:56.521 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:56.512+0000 7faa117aa8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:05:56.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:56.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:56.814 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:05:56.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:05:56.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:56.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:05:56.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:05:57.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:05:57.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:05:57.976 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:05:57.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:05:57.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:05:57.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:05:58.132 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1445037759,v1:127.0.0.1:6803/1445037759] [v2:127.0.0.1:6804/1445037759,v1:127.0.0.1:6805/1445037759] exists,up 9544989c-8f90-45d0-9efe-5ab41b5e7f5c 2026-02-17T12:05:58.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:05:58.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:05:58.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:05:58.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:05:58.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:923: list_missing_erasure_coded: run_osd td/osd-scrub-repair 1 2026-02-17T12:05:58.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:05:58.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:05:58.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:05:58.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:05:58.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:05:58.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:05:58.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:05:58.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:05:58.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:05:58.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:05:58.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:05:58.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:05:58.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:05:58.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:05:58.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:05:58.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:05:58.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:05:58.138 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:05:58.138 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:05:58.138 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:05:58.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:05:58.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:05:58.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:05:58.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:05:58.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:05:58.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:05:58.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:05:58.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:05:58.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:05:58.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:05:58.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:05:58.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:05:58.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:05:58.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:05:58.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:05:58.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=38f68e84-f83b-4bbb-b0b2-4a6861121090 2026-02-17T12:05:58.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 38f68e84-f83b-4bbb-b0b2-4a6861121090' 2026-02-17T12:05:58.140 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 38f68e84-f83b-4bbb-b0b2-4a6861121090 2026-02-17T12:05:58.141 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:05:58.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCmWZRpNpu7CBAA8y5raCNKMNajQxAFAh3scw== 2026-02-17T12:05:58.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCmWZRpNpu7CBAA8y5raCNKMNajQxAFAh3scw=="}' 2026-02-17T12:05:58.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 38f68e84-f83b-4bbb-b0b2-4a6861121090 -i td/osd-scrub-repair/1/new.json 2026-02-17T12:05:58.307 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:05:58.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T12:05:58.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCmWZRpNpu7CBAA8y5raCNKMNajQxAFAh3scw== --osd-uuid 38f68e84-f83b-4bbb-b0b2-4a6861121090 2026-02-17T12:05:58.334 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:58.328+0000 7fc0b8bdf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:58.336 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:58.328+0000 7fc0b8bdf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:58.337 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:58.328+0000 7fc0b8bdf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:05:58.337 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:58.332+0000 7fc0b8bdf8c0 -1 bdev(0x56508f8fbc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:05:58.337 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:05:58.332+0000 7fc0b8bdf8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T12:06:00.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T12:06:00.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:06:00.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:06:00.604 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:06:00.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:06:00.797 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:06:00.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:06:00.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:06:00.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:06:00.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:06:00.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:06:00.820 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:00.812+0000 7f298376e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:00.822 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:00.816+0000 7f298376e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:00.823 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:00.816+0000 7f298376e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:00.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:06:00.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:06:00.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:06:00.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:06:00.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:06:00.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:00.951 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:06:00.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:06:00.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:00.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:06:01.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:02.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:02.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:02.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:06:02.100 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:06:02.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:02.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:06:02.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:02.343 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:02.336+0000 7f298376e8c0 -1 Falling back to public interface 2026-02-17T12:06:03.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:03.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:03.258 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:06:03.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:06:03.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:03.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:06:03.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:04.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:04.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:04.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:06:04.409 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:06:04.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:04.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:06:04.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:05.329 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:05.320+0000 7f298376e8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:06:05.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:05.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:05.569 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:06:05.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:06:05.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:05.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:06:05.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:06.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:06.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:06.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:06:06.727 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:06:06.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:06.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:06:06.884 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/601147055,v1:127.0.0.1:6811/601147055] [v2:127.0.0.1:6812/601147055,v1:127.0.0.1:6813/601147055] exists,up 38f68e84-f83b-4bbb-b0b2-4a6861121090 2026-02-17T12:06:06.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:06:06.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:06:06.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:06:06.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:922: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:06:06.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:923: list_missing_erasure_coded: run_osd td/osd-scrub-repair 2 2026-02-17T12:06:06.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:06:06.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:06:06.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:06:06.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:06:06.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T12:06:06.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:06:06.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:06:06.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:06:06.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:06:06.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T12:06:06.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T12:06:06.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:06:06.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:06:06.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:06:06.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:06:06.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:06:06.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:06:06.890 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:06:06.891 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:06:06.891 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:06:06.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:06:06.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:06:06.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:06:06.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:06:06.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:06:06.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:06:06.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:06:06.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:06:06.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:06:06.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:06:06.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:06:06.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:06:06.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:06:06.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T12:06:06.893 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:06:06.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=bdc8737e-f41f-460d-a21a-c20877414d89 2026-02-17T12:06:06.893 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 bdc8737e-f41f-460d-a21a-c20877414d89 2026-02-17T12:06:06.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 bdc8737e-f41f-460d-a21a-c20877414d89' 2026-02-17T12:06:06.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:06:06.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCuWZRp4xeLNRAA3540ivyD/A2HCfLVIIdjDQ== 2026-02-17T12:06:06.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCuWZRp4xeLNRAA3540ivyD/A2HCfLVIIdjDQ=="}' 2026-02-17T12:06:06.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new bdc8737e-f41f-460d-a21a-c20877414d89 -i td/osd-scrub-repair/2/new.json 2026-02-17T12:06:07.062 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:06:07.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T12:06:07.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCuWZRp4xeLNRAA3540ivyD/A2HCfLVIIdjDQ== --osd-uuid bdc8737e-f41f-460d-a21a-c20877414d89 2026-02-17T12:06:07.088 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:07.080+0000 7f093b59c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:07.090 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:07.084+0000 7f093b59c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:07.091 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:07.084+0000 7f093b59c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:07.092 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:07.084+0000 7f093b59c8c0 -1 bdev(0x5580a9837c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:06:07.092 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:07.084+0000 7f093b59c8c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T12:06:09.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T12:06:09.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:06:09.352 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T12:06:09.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:06:09.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:06:09.541 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T12:06:09.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:06:09.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:06:09.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:06:09.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:06:09.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:06:09.565 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:09.556+0000 7f5d027398c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:09.566 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:09.560+0000 7f5d027398c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:09.568 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:09.560+0000 7f5d027398c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:09.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:06:09.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:06:09.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:06:09.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:06:09.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:06:09.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:09.700 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:06:09.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:06:09.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:09.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:06:09.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:10.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:10.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:10.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:06:10.859 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:06:10.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:10.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:06:11.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:11.095 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:11.088+0000 7f5d027398c0 -1 Falling back to public interface 2026-02-17T12:06:12.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:12.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:12.013 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:06:12.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:06:12.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:12.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:06:12.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:13.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:13.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:13.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:06:13.170 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:06:13.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:13.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:06:13.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:14.122 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:14.116+0000 7f5d027398c0 -1 osd.2 0 log_to_monitors true 2026-02-17T12:06:14.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:14.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:14.325 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:06:14.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:06:14.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:14.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:06:14.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:15.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:15.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:15.473 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:06:15.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:06:15.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:15.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:06:15.627 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3633287042,v1:127.0.0.1:6819/3633287042] [v2:127.0.0.1:6820/3633287042,v1:127.0.0.1:6821/3633287042] exists,up bdc8737e-f41f-460d-a21a-c20877414d89 2026-02-17T12:06:15.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:06:15.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:06:15.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:06:15.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:925: list_missing_erasure_coded: create_rbd_pool 2026-02-17T12:06:15.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T12:06:15.769 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' does not exist 2026-02-17T12:06:15.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T12:06:15.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:06:15.993 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T12:06:16.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:06:17.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T12:06:17.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:926: list_missing_erasure_coded: wait_for_clean 2026-02-17T12:06:17.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:06:17.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:06:17.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:06:17.287 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:06:17.288 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:06:17.288 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:06:17.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:06:17.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:06:17.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:06:17.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:06:17.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:06:17.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:06:17.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:06:17.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:06:17.411 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:06:17.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:06:17.562 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:06:17.562 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:06:17.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:06:17.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:17.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:06:17.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T12:06:17.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T12:06:17.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T12:06:17.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:17.635 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:06:17.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T12:06:17.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T12:06:17.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-17T12:06:17.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:17.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:06:17.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509442 2026-02-17T12:06:17.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509442 2026-02-17T12:06:17.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-64424509442' 2026-02-17T12:06:17.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:17.780 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T12:06:17.780 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:17.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:06:17.782 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T12:06:17.782 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:17.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T12:06:17.784 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836485 2026-02-17T12:06:17.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T12:06:17.784 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:06:17.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T12:06:17.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:06:18.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:06:18.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:06:19.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T12:06:19.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:06:20.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:06:20.090 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:06:20.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T12:06:20.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:20.247 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T12:06:20.247 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:20.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:06:20.250 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T12:06:20.250 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:20.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T12:06:20.252 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672964 2026-02-17T12:06:20.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T12:06:20.253 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:06:20.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T12:06:20.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:20.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509442 2026-02-17T12:06:20.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:20.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:06:20.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509442 2026-02-17T12:06:20.413 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:20.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509442 2026-02-17T12:06:20.415 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 64424509442 2026-02-17T12:06:20.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509442' 2026-02-17T12:06:20.415 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:06:20.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509442 -lt 64424509442 2026-02-17T12:06:20.572 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:06:20.573 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:06:20.573 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:06:20.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T12:06:20.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:06:20.760 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:06:20.760 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:06:20.760 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:06:20.760 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:06:20.760 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:06:20.760 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:06:20.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T12:06:20.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:06:20.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:06:20.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:06:21.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T12:06:21.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:06:21.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:06:21.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:928: list_missing_erasure_coded: create_ec_pool ecpool true k=2 m=1 2026-02-17T12:06:21.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T12:06:21.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T12:06:21.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=true 2026-02-17T12:06:21.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T12:06:21.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-17T12:06:21.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T12:06:21.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T12:06:21.612 INFO:tasks.workunit.client.0.trial128.stderr:pool 'ecpool' created 2026-02-17T12:06:21.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:06:22.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' true = true ']' 2026-02-17T12:06:22.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2508: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-17T12:06:22.873 INFO:tasks.workunit.client.0.trial128.stderr:set pool 2 allow_ec_overwrites to true 2026-02-17T12:06:22.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T12:06:22.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:06:22.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:06:22.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:06:22.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:06:22.885 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:06:22.885 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:06:22.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:06:22.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:06:22.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:06:22.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:06:22.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:06:22.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:06:22.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:06:22.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:06:22.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:06:23.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:06:23.153 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:06:23.153 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:06:23.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:06:23.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:23.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:06:23.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836488 2026-02-17T12:06:23.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836488 2026-02-17T12:06:23.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488' 2026-02-17T12:06:23.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:23.225 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:06:23.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672966 2026-02-17T12:06:23.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672966 2026-02-17T12:06:23.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-42949672966' 2026-02-17T12:06:23.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:23.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:06:23.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509444 2026-02-17T12:06:23.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509444 2026-02-17T12:06:23.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-42949672966 2-64424509444' 2026-02-17T12:06:23.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:23.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836488 2026-02-17T12:06:23.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:23.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:06:23.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836488 2026-02-17T12:06:23.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:23.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836488 2026-02-17T12:06:23.373 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836488 2026-02-17T12:06:23.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-17T12:06:23.374 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:06:23.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-17T12:06:23.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:23.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672966 2026-02-17T12:06:23.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:23.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:06:23.534 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672966 2026-02-17T12:06:23.534 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:23.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672966 2026-02-17T12:06:23.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2026-02-17T12:06:23.536 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672966 2026-02-17T12:06:23.537 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:06:23.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672966 -lt 42949672966 2026-02-17T12:06:23.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:23.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509444 2026-02-17T12:06:23.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:23.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:06:23.703 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509444 2026-02-17T12:06:23.703 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:23.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509444 2026-02-17T12:06:23.705 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 64424509444 2026-02-17T12:06:23.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509444' 2026-02-17T12:06:23.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:06:23.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509443 -lt 64424509444 2026-02-17T12:06:23.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:06:24.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:06:24.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:06:25.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509443 -lt 64424509444 2026-02-17T12:06:25.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:06:26.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:06:26.027 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:06:26.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509445 -lt 64424509444 2026-02-17T12:06:26.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:06:26.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:06:26.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:06:26.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:06:26.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:06:26.362 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:06:26.362 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:06:26.362 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:06:26.362 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:06:26.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:06:26.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:06:26.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:06:26.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:06:26.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:06:26.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:06:26.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:06:26.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:06:26.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:06:26.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T12:06:26.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:931: list_missing_erasure_coded: add_something td/osd-scrub-repair ecpool MOBJ0 2026-02-17T12:06:26.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T12:06:26.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T12:06:26.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=MOBJ0 2026-02-17T12:06:26.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T12:06:26.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T12:06:26.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T12:06:26.890 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:06:26.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T12:06:27.096 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:06:27.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T12:06:27.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T12:06:27.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put MOBJ0 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:06:27.129 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:932: list_missing_erasure_coded: get_osds ecpool MOBJ0 2026-02-17T12:06:27.129 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T12:06:27.129 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=MOBJ0 2026-02-17T12:06:27.130 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool MOBJ0 2026-02-17T12:06:27.130 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:06:27.287 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=2 2026-02-17T12:06:27.287 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:06:27.287 INFO:tasks.workunit.client.0.trial128.stderr:0' 2026-02-17T12:06:27.287 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 2 1 0 2026-02-17T12:06:27.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:932: list_missing_erasure_coded: osds0=('2' '1' '0') 2026-02-17T12:06:27.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:932: list_missing_erasure_coded: local -a osds0 2026-02-17T12:06:27.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:935: list_missing_erasure_coded: add_something td/osd-scrub-repair ecpool MOBJ1 2026-02-17T12:06:27.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T12:06:27.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T12:06:27.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=MOBJ1 2026-02-17T12:06:27.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T12:06:27.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T12:06:27.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T12:06:27.499 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:06:27.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T12:06:27.702 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:06:27.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T12:06:27.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T12:06:27.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put MOBJ1 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:06:27.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:936: list_missing_erasure_coded: get_osds ecpool MOBJ1 2026-02-17T12:06:27.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T12:06:27.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=MOBJ1 2026-02-17T12:06:27.735 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool MOBJ1 2026-02-17T12:06:27.735 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:06:27.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=2 2026-02-17T12:06:27.885 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:06:27.885 INFO:tasks.workunit.client.0.trial128.stderr:0' 2026-02-17T12:06:27.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 2 1 0 2026-02-17T12:06:27.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:936: list_missing_erasure_coded: osds1=('2' '1' '0') 2026-02-17T12:06:27.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:936: list_missing_erasure_coded: local -a osds1 2026-02-17T12:06:27.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: list_missing_erasure_coded: seq 0 2 2026-02-17T12:06:27.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:06:27.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:940: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T12:06:27.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:06:27.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:06:27.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:06:27.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:06:27.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:06:29.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:06:29.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:06:29.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:940: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:06:29.201 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:06:29.201 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:06:29.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:06:29.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:06:29.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:06:30.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:06:30.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:939: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:06:30.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:940: list_missing_erasure_coded: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T12:06:30.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:06:30.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:06:30.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:06:30.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:06:30.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:06:31.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:06:31.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:943: list_missing_erasure_coded: id=2 2026-02-17T12:06:31.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:944: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 MOBJ0 remove 2026-02-17T12:06:32.471 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#2:a9030505:::MOBJ0:head# 2026-02-17T12:06:33.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:946: list_missing_erasure_coded: id=1 2026-02-17T12:06:33.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:947: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 MOBJ0 remove 2026-02-17T12:06:33.652 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#2:a9030505:::MOBJ0:head# 2026-02-17T12:06:34.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:950: list_missing_erasure_coded: id=1 2026-02-17T12:06:34.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:951: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 MOBJ1 remove 2026-02-17T12:06:34.822 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#2:8dd16d55:::MOBJ1:head# 2026-02-17T12:06:35.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:953: list_missing_erasure_coded: id=0 2026-02-17T12:06:35.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:954: list_missing_erasure_coded: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 MOBJ1 remove 2026-02-17T12:06:35.998 INFO:tasks.workunit.client.0.trial128.stdout:remove 2#2:8dd16d55:::MOBJ1:head# 2026-02-17T12:06:36.536 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: list_missing_erasure_coded: seq 0 2 2026-02-17T12:06:36.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:06:36.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:958: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 0 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:06:36.538 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:06:36.539 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:06:36.539 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:06:36.539 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:06:36.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:06:36.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:06:36.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:06:36.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:06:36.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:06:36.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:06:36.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:06:36.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:06:36.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:06:36.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:06:36.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:06:36.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:06:36.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:06:36.541 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T12:06:36.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:06:36.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T12:06:36.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:06:36.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:06:36.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:06:36.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:06:36.558 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:36.551+0000 7fde9d2248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:36.559 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:36.551+0000 7fde9d2248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:36.561 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:36.551+0000 7fde9d2248c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:36.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:06:36.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:06:36.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:06:36.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:06:36.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:06:36.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:36.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:06:36.700 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:06:36.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:36.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:06:36.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:37.787 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:37.779+0000 7fde9d2248c0 -1 Falling back to public interface 2026-02-17T12:06:37.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:37.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:37.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:06:37.854 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:06:37.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:37.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:06:38.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:39.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:39.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:39.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:06:39.004 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:06:39.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:39.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:06:39.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:40.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:40.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:40.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:06:40.155 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:06:40.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:40.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:06:40.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:40.782 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:40.775+0000 7fde9d2248c0 -1 osd.0 33 log_to_monitors true 2026-02-17T12:06:41.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:41.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:41.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:06:41.304 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:06:41.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:41.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:06:41.456 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 40 up_thru 40 down_at 34 last_clean_interval [5,33) [v2:127.0.0.1:6802/1094530970,v1:127.0.0.1:6803/1094530970] [v2:127.0.0.1:6804/1094530970,v1:127.0.0.1:6805/1094530970] exists,up 9544989c-8f90-45d0-9efe-5ab41b5e7f5c 2026-02-17T12:06:41.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:06:41.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:06:41.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:06:41.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:06:41.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:958: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 1 2026-02-17T12:06:41.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:06:41.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:06:41.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:06:41.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:06:41.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:06:41.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:06:41.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:06:41.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:06:41.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:06:41.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:06:41.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:06:41.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:06:41.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:06:41.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:06:41.458 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:06:41.458 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:06:41.458 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:06:41.459 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:06:41.459 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:06:41.459 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:06:41.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:06:41.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:06:41.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:06:41.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:06:41.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:06:41.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:06:41.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:06:41.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:06:41.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:06:41.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:06:41.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:06:41.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:06:41.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:06:41.461 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:06:41.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:06:41.461 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:06:41.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:06:41.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:06:41.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:06:41.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:06:41.474 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:41.467+0000 7fd6a29978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:41.476 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:41.467+0000 7fd6a29978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:41.477 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:41.467+0000 7fd6a29978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:41.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:06:41.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:06:41.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:06:41.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:06:41.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:06:41.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:41.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:06:41.618 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:06:41.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:41.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:06:41.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:42.707 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:42.699+0000 7fd6a29978c0 -1 Falling back to public interface 2026-02-17T12:06:42.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:42.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:42.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:06:42.771 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:06:42.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:42.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:06:42.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:43.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:43.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:43.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:06:43.923 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:06:43.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:43.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:06:44.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:45.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:45.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:45.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:06:45.073 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:06:45.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:45.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:06:45.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:45.682 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:45.675+0000 7fd6a29978c0 -1 osd.1 35 log_to_monitors true 2026-02-17T12:06:46.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:46.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:46.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:06:46.227 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:06:46.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:46.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:06:46.379 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 43 up_thru 43 down_at 36 last_clean_interval [10,35) [v2:127.0.0.1:6810/2648430275,v1:127.0.0.1:6811/2648430275] [v2:127.0.0.1:6812/2648430275,v1:127.0.0.1:6813/2648430275] exists,up 38f68e84-f83b-4bbb-b0b2-4a6861121090 2026-02-17T12:06:46.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:06:46.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:06:46.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:06:46.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:957: list_missing_erasure_coded: for id in $(seq 0 2) 2026-02-17T12:06:46.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:958: list_missing_erasure_coded: activate_osd td/osd-scrub-repair 2 2026-02-17T12:06:46.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:06:46.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:06:46.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:06:46.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:06:46.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T12:06:46.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:06:46.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:06:46.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:06:46.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:06:46.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T12:06:46.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T12:06:46.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:06:46.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:06:46.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:06:46.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:06:46.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:06:46.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:06:46.381 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:06:46.381 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:06:46.381 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:06:46.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:06:46.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:06:46.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:06:46.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:06:46.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:06:46.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:06:46.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:06:46.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:06:46.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:06:46.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:06:46.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:06:46.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T12:06:46.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:06:46.382 INFO:tasks.workunit.client.0.trial128.stderr:start osd.2 2026-02-17T12:06:46.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:06:46.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T12:06:46.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:06:46.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:06:46.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:06:46.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:06:46.397 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:46.387+0000 7fe56ac128c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:46.399 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:46.391+0000 7fe56ac128c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:46.400 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:46.391+0000 7fe56ac128c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:06:46.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:06:46.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:06:46.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:06:46.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:06:46.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:06:46.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:46.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:06:46.538 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:06:46.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:46.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:06:46.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:47.635 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:47.627+0000 7fe56ac128c0 -1 Falling back to public interface 2026-02-17T12:06:47.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:47.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:47.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:06:47.689 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:06:47.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:47.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:06:47.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:48.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:48.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:48.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:06:48.837 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:06:48.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:48.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:06:48.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:49.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:49.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:49.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:06:49.988 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:06:49.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:49.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:06:50.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:06:51.103 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:06:51.095+0000 7fe56ac128c0 -1 osd.2 37 log_to_monitors true 2026-02-17T12:06:51.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:06:51.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:06:51.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:06:51.139 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:06:51.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:06:51.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:06:51.287 INFO:tasks.workunit.client.0.trial128.stderr:osd.2 up in weight 1 up_from 47 up_thru 36 down_at 38 last_clean_interval [15,37) [v2:127.0.0.1:6818/1697259103,v1:127.0.0.1:6819/1697259103] [v2:127.0.0.1:6820/1697259103,v1:127.0.0.1:6821/1697259103] exists,up bdc8737e-f41f-460d-a21a-c20877414d89 2026-02-17T12:06:51.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:06:51.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:06:51.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:06:51.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:960: list_missing_erasure_coded: create_rbd_pool 2026-02-17T12:06:51.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T12:06:51.464 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' removed 2026-02-17T12:06:51.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T12:06:51.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:06:51.669 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T12:06:51.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:06:52.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T12:06:52.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:961: list_missing_erasure_coded: wait_for_clean 2026-02-17T12:06:52.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:06:52.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:06:52.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:06:52.964 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:06:52.965 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:06:52.965 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:06:52.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:06:52.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:06:52.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:06:53.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:06:53.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:06:53.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:06:53.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:06:53.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:06:53.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:06:53.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:06:53.225 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:06:53.225 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:06:53.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:06:53.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:53.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:06:53.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691844 2026-02-17T12:06:53.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691844 2026-02-17T12:06:53.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-171798691844' 2026-02-17T12:06:53.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:53.299 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:06:53.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=184683593731 2026-02-17T12:06:53.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 184683593731 2026-02-17T12:06:53.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-171798691844 1-184683593731' 2026-02-17T12:06:53.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:06:53.367 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:06:53.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=201863462914 2026-02-17T12:06:53.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 201863462914 2026-02-17T12:06:53.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-171798691844 1-184683593731 2-201863462914' 2026-02-17T12:06:53.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:53.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-171798691844 2026-02-17T12:06:53.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:53.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:06:53.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-171798691844 2026-02-17T12:06:53.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:53.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691844 2026-02-17T12:06:53.442 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 171798691844 2026-02-17T12:06:53.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 171798691844' 2026-02-17T12:06:53.443 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:06:53.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691844 -lt 171798691844 2026-02-17T12:06:53.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:53.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-184683593731 2026-02-17T12:06:53.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:53.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:06:53.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-184683593731 2026-02-17T12:06:53.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:53.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=184683593731 2026-02-17T12:06:53.605 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 184683593731 2026-02-17T12:06:53.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 184683593731' 2026-02-17T12:06:53.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:06:53.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 184683593730 -lt 184683593731 2026-02-17T12:06:53.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:06:54.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:06:54.767 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:06:54.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 184683593730 -lt 184683593731 2026-02-17T12:06:54.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:06:55.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:06:55.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:06:56.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 184683593731 -lt 184683593731 2026-02-17T12:06:56.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:06:56.079 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-201863462914 2026-02-17T12:06:56.079 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:06:56.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:06:56.081 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-201863462914 2026-02-17T12:06:56.082 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:06:56.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=201863462914 2026-02-17T12:06:56.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 201863462914' 2026-02-17T12:06:56.084 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 201863462914 2026-02-17T12:06:56.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:06:56.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 201863462914 -lt 201863462914 2026-02-17T12:06:56.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:06:56.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:06:56.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:06:56.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:06:56.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:06:56.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:06:56.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:06:56.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:06:56.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:06:56.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:06:56.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:06:56.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:06:56.587 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:06:56.588 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:06:56.588 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:06:56.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:06:56.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:06:56.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:06:56.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:964: list_missing_erasure_coded: get_pg ecpool MOBJ0 2026-02-17T12:06:56.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T12:06:56.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=MOBJ0 2026-02-17T12:06:56.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool MOBJ0 2026-02-17T12:06:56.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T12:06:56.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:964: list_missing_erasure_coded: local pg=2.0 2026-02-17T12:06:56.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:968: list_missing_erasure_coded: repair 2.0 2026-02-17T12:06:56.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=2.0 2026-02-17T12:06:56.929 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 2.0 2026-02-17T12:06:56.929 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:06:56.929 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:06:56.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:06:56.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:06:57.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T12:06:21.607182+0000 2026-02-17T12:06:57.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 2.0 2026-02-17T12:06:57.221 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0s0 on osd.2 to repair 2026-02-17T12:06:57.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 2.0 2026-02-17T12:06:21.607182+0000 2026-02-17T12:06:57.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T12:06:57.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:06:21.607182+0000 2026-02-17T12:06:57.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:06:57.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:06:57.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:06:57.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:06:57.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:06:57.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:06:57.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:06:57.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:06:57.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:06:21.607182+0000 '>' 2026-02-17T12:06:21.607182+0000 2026-02-17T12:06:57.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:06:58.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:06:58.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:06:58.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:06:58.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:06:58.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:06:58.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:06:58.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:06:58.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:06:21.607182+0000 '>' 2026-02-17T12:06:21.607182+0000 2026-02-17T12:06:58.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:06:59.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:06:59.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:06:59.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:06:59.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:06:59.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:06:59.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:06:59.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:06:59.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:06:21.607182+0000 '>' 2026-02-17T12:06:21.607182+0000 2026-02-17T12:06:59.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:07:00.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:07:00.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:07:00.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:07:00.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:07:00.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:07:00.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:07:00.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:07:00.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:06:21.607182+0000 '>' 2026-02-17T12:06:21.607182+0000 2026-02-17T12:07:00.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:07:01.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:07:01.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:07:01.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:07:01.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:07:01.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:07:01.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:07:01.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:07:02.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:06:58.110330+0000 '>' 2026-02-17T12:06:21.607182+0000 2026-02-17T12:07:02.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:07:02.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:970: list_missing_erasure_coded: seq 0 120 2026-02-17T12:07:02.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:970: list_missing_erasure_coded: for i in $(seq 0 120) 2026-02-17T12:07:02.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:971: list_missing_erasure_coded: '[' 0 -lt 60 ']' 2026-02-17T12:07:02.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:972: list_missing_erasure_coded: ceph pg 2.0 list_unfound 2026-02-17T12:07:02.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:972: list_missing_erasure_coded: egrep 'MOBJ0|MOBJ1' 2026-02-17T12:07:02.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:972: list_missing_erasure_coded: wc -l 2026-02-17T12:07:02.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:972: list_missing_erasure_coded: matches=2 2026-02-17T12:07:02.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:973: list_missing_erasure_coded: '[' 2 -eq 2 ']' 2026-02-17T12:07:02.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:973: list_missing_erasure_coded: break 2026-02-17T12:07:02.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T12:07:02.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:07:02.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:07:02.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:07:02.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:07:02.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:07:02.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:07:02.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:07:02.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:07:02.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:07:02.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:07:02.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:07:02.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:07:02.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:07:02.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:07:02.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:07:02.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:07:02.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:07:02.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:07:02.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:07:02.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:07:02.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:07:02.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:07:02.423 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:07:02.424 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:07:02.424 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:07:02.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:07:02.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:07:02.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:07:02.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T12:07:02.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T12:07:02.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T12:07:02.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T12:07:02.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:07:02.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:07:02.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:07:02.427 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:07:02.427 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:07:02.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:07:02.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:07:02.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:07:02.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:07:02.430 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:07:02.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:07:02.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:07:02.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:07:02.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:07:02.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:07:02.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:07:02.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:07:02.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:07:02.435 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:07:02.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:07:02.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:07:02.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:07:02.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:07:02.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:07:02.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:07:02.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:07:02.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:07:02.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:07:02.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T12:07:02.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:07:02.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:07:02.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:07:02.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T12:07:02.444 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:07:02.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:07:02.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:07:02.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T12:07:02.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_periodic_scrub_replicated td/osd-scrub-repair 2026-02-17T12:07:02.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5759: TEST_periodic_scrub_replicated: local dir=td/osd-scrub-repair 2026-02-17T12:07:02.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5760: TEST_periodic_scrub_replicated: local poolname=psr_pool 2026-02-17T12:07:02.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5761: TEST_periodic_scrub_replicated: local objname=POBJ 2026-02-17T12:07:02.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5763: TEST_periodic_scrub_replicated: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-17T12:07:02.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T12:07:02.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:07:02.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:07:02.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:07:02.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T12:07:02.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-17T12:07:02.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:07:02.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:07:02.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:07:02.467 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:07:02.468 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:07:02.468 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:07:02.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:07:02.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T12:07:02.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:07:02.498 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:07:02.498 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:07:02.498 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:07:02.498 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:07:02.499 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:07:02.499 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:07:02.500 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:07:02.500 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:07:02.500 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:07:02.501 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:07:02.501 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:07:02.501 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:07:02.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:07:02.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T12:07:02.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:07:02.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:07:02.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:07:02.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:07:02.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:07:02.559 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:07:02.559 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:07:02.559 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:07:02.559 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:07:02.560 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:07:02.560 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:07:02.560 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:07:02.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:07:02.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T12:07:02.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5764: TEST_periodic_scrub_replicated: run_mgr td/osd-scrub-repair x 2026-02-17T12:07:02.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T12:07:02.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:07:02.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:07:02.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:07:02.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T12:07:02.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:07:02.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:07:02.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:07:02.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:07:02.711 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:07:02.711 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:07:02.711 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:07:02.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:07:02.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:07:02.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:07:02.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5765: TEST_periodic_scrub_replicated: local 'ceph_osd_args=--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 ' 2026-02-17T12:07:02.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5766: TEST_periodic_scrub_replicated: ceph_osd_args+=--osd_scrub_backoff_ratio=0 2026-02-17T12:07:02.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5767: TEST_periodic_scrub_replicated: run_osd td/osd-scrub-repair 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-17T12:07:02.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:07:02.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:07:02.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:07:02.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:07:02.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:07:02.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:07:02.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:07:02.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:07:02.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:07:02.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:07:02.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:07:02.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:07:02.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:07:02.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:07:02.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:07:02.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:07:02.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:07:02.739 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:07:02.739 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:07:02.739 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:07:02.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:07:02.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:07:02.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:07:02.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:07:02.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:07:02.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:07:02.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:07:02.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:07:02.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:07:02.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:07:02.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:07:02.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:07:02.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0' 2026-02-17T12:07:02.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:07:02.741 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:07:02.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3f07edf3-015f-4490-9564-1f69933cc802 2026-02-17T12:07:02.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 3f07edf3-015f-4490-9564-1f69933cc802' 2026-02-17T12:07:02.742 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 3f07edf3-015f-4490-9564-1f69933cc802 2026-02-17T12:07:02.742 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:07:02.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDmWZRpEXdfLBAA8vnBtS7MlLRj8/fyIpndaQ== 2026-02-17T12:07:02.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDmWZRpEXdfLBAA8vnBtS7MlLRj8/fyIpndaQ=="}' 2026-02-17T12:07:02.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3f07edf3-015f-4490-9564-1f69933cc802 -i td/osd-scrub-repair/0/new.json 2026-02-17T12:07:02.840 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:07:02.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T12:07:02.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --mkfs --key AQDmWZRpEXdfLBAA8vnBtS7MlLRj8/fyIpndaQ== --osd-uuid 3f07edf3-015f-4490-9564-1f69933cc802 2026-02-17T12:07:02.865 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:02.855+0000 7f50074718c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:07:02.866 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:02.859+0000 7f50074718c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:07:02.867 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:02.859+0000 7f50074718c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:07:02.868 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:02.859+0000 7f50074718c0 -1 bdev(0x556e69fe3c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:07:02.868 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:02.859+0000 7f50074718c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T12:07:05.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T12:07:05.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:07:05.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:07:05.657 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:07:05.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:07:05.855 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:07:05.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:07:05.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-17T12:07:05.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:07:05.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:07:05.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:07:05.879 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:05.871+0000 7f1ce73178c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:07:05.881 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:05.871+0000 7f1ce73178c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:07:05.882 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:05.875+0000 7f1ce73178c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:07:06.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:07:06.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:07:06.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:07:06.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:07:06.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:07:06.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:06.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:07:06.009 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:07:06.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:06.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:07:06.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:07:07.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:07:07.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:07.165 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:07:07.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:07:07.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:07.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:07:07.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:07:07.371 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:07.363+0000 7f1ce73178c0 -1 Falling back to public interface 2026-02-17T12:07:08.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:07:08.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:08.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:07:08.321 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:07:08.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:08.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:07:08.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:07:09.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:07:09.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:09.481 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:07:09.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:07:09.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:09.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:07:09.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:07:10.357 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:10.346+0000 7f1ce73178c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:07:10.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:07:10.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:10.639 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:07:10.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:07:10.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:10.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:07:10.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:07:11.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:07:11.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:11.792 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:07:11.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:07:11.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:11.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:07:11.950 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/793742193,v1:127.0.0.1:6803/793742193] [v2:127.0.0.1:6804/793742193,v1:127.0.0.1:6805/793742193] exists,up 3f07edf3-015f-4490-9564-1f69933cc802 2026-02-17T12:07:11.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:07:11.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:07:11.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:07:11.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5768: TEST_periodic_scrub_replicated: run_osd td/osd-scrub-repair 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-17T12:07:11.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:07:11.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:07:11.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:07:11.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:07:11.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:07:11.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:07:11.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:07:11.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:07:11.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:07:11.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:07:11.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:07:11.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:07:11.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:07:11.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:07:11.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:07:11.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:07:11.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:07:11.955 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:07:11.956 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:07:11.956 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:07:11.956 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:07:11.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:07:11.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:07:11.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:07:11.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:07:11.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:07:11.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:07:11.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:07:11.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:07:11.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:07:11.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:07:11.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:07:11.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0' 2026-02-17T12:07:11.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:07:11.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:07:11.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=36754adc-216b-4960-b2d7-03cc3942a486 2026-02-17T12:07:11.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 36754adc-216b-4960-b2d7-03cc3942a486' 2026-02-17T12:07:11.958 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 36754adc-216b-4960-b2d7-03cc3942a486 2026-02-17T12:07:11.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:07:11.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDvWZRpjshrORAAJTSVLduU/+TrQbPSH/aiaA== 2026-02-17T12:07:11.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDvWZRpjshrORAAJTSVLduU/+TrQbPSH/aiaA=="}' 2026-02-17T12:07:11.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 36754adc-216b-4960-b2d7-03cc3942a486 -i td/osd-scrub-repair/1/new.json 2026-02-17T12:07:12.128 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:07:12.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T12:07:12.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --mkfs --key AQDvWZRpjshrORAAJTSVLduU/+TrQbPSH/aiaA== --osd-uuid 36754adc-216b-4960-b2d7-03cc3942a486 2026-02-17T12:07:12.156 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:12.146+0000 7f093f3e78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:07:12.158 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:12.150+0000 7f093f3e78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:07:12.159 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:12.150+0000 7f093f3e78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:07:12.159 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:12.150+0000 7f093f3e78c0 -1 bdev(0x558da7249c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:07:12.159 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:12.150+0000 7f093f3e78c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T12:07:14.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T12:07:14.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:07:14.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:07:14.437 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:07:14.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:07:14.636 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:07:14.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:07:14.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-17T12:07:14.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:07:14.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:07:14.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:07:14.653 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:14.642+0000 7fc0126c58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:07:14.654 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:14.646+0000 7fc0126c58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:07:14.655 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:14.646+0000 7fc0126c58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:07:14.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:07:14.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:07:14.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:07:14.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:07:14.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:07:14.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:14.790 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:07:14.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:07:14.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:14.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:07:14.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:07:15.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:07:15.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:15.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:07:15.948 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:07:15.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:15.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:07:16.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:07:16.183 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:16.174+0000 7fc0126c58c0 -1 Falling back to public interface 2026-02-17T12:07:17.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:07:17.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:17.103 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:07:17.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:07:17.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:17.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:07:17.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:07:18.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:07:18.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:18.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:07:18.255 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:07:18.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:18.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:07:18.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:07:19.200 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:19.190+0000 7fc0126c58c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:07:19.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:07:19.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:19.412 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:07:19.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:07:19.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:19.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:07:19.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:07:20.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:07:20.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:20.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:07:20.562 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:07:20.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:20.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:07:20.716 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2391577228,v1:127.0.0.1:6811/2391577228] [v2:127.0.0.1:6812/2391577228,v1:127.0.0.1:6813/2391577228] exists,up 36754adc-216b-4960-b2d7-03cc3942a486 2026-02-17T12:07:20.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:07:20.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:07:20.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:07:20.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5769: TEST_periodic_scrub_replicated: create_rbd_pool 2026-02-17T12:07:20.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T12:07:20.859 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' does not exist 2026-02-17T12:07:20.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T12:07:20.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:07:21.113 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T12:07:21.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:07:22.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T12:07:22.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5770: TEST_periodic_scrub_replicated: wait_for_clean 2026-02-17T12:07:22.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:07:22.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:07:22.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:07:22.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:07:22.410 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:07:22.410 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:07:22.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:07:22.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:07:22.411 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:07:22.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:07:22.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:07:22.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:07:22.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:07:22.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:07:22.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:07:22.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:07:22.685 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:07:22.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:07:22.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:22.686 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:07:22.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T12:07:22.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T12:07:22.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T12:07:22.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:22.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:07:22.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705666 2026-02-17T12:07:22.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705666 2026-02-17T12:07:22.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-38654705666' 2026-02-17T12:07:22.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:22.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T12:07:22.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:22.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:07:22.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T12:07:22.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:22.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T12:07:22.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T12:07:22.833 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836484 2026-02-17T12:07:22.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:22.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T12:07:22.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:07:24.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:07:24.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:24.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T12:07:24.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:24.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705666 2026-02-17T12:07:24.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:24.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:07:24.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705666 2026-02-17T12:07:24.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:24.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705666 2026-02-17T12:07:24.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705666' 2026-02-17T12:07:24.155 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705666 2026-02-17T12:07:24.156 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:07:24.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705666 -lt 38654705666 2026-02-17T12:07:24.315 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:07:24.315 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:07:24.315 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:07:24.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T12:07:24.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:07:24.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:07:24.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:07:24.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:07:24.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:07:24.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:07:24.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:07:24.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T12:07:24.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:07:24.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:07:24.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:07:24.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T12:07:24.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:07:24.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:07:24.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5772: TEST_periodic_scrub_replicated: create_pool psr_pool 1 1 2026-02-17T12:07:24.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool 1 1 2026-02-17T12:07:25.091 INFO:tasks.workunit.client.0.trial128.stderr:pool 'psr_pool' created 2026-02-17T12:07:25.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:07:26.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5773: TEST_periodic_scrub_replicated: wait_for_clean 2026-02-17T12:07:26.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:07:26.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:07:26.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:07:26.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:07:26.107 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:07:26.107 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:07:26.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:07:26.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:07:26.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:07:26.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:07:26.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:07:26.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:07:26.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:07:26.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:07:26.229 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:07:26.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:07:26.389 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:07:26.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:07:26.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:26.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:07:26.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T12:07:26.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T12:07:26.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T12:07:26.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:26.461 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:07:26.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705668 2026-02-17T12:07:26.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705668 2026-02-17T12:07:26.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705668' 2026-02-17T12:07:26.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:26.533 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T12:07:26.534 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:26.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:07:26.535 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T12:07:26.536 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:26.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T12:07:26.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T12:07:26.538 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836485 2026-02-17T12:07:26.538 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:26.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T12:07:26.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:07:27.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:07:27.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:27.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T12:07:27.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:27.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705668 2026-02-17T12:07:27.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:27.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:07:27.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705668 2026-02-17T12:07:27.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:27.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705668 2026-02-17T12:07:27.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-17T12:07:27.862 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705668 2026-02-17T12:07:27.862 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:07:28.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-17T12:07:28.025 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:07:28.025 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:07:28.025 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:07:28.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:07:28.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:07:28.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:07:28.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:07:28.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:07:28.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:07:28.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:07:28.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:07:28.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:07:28.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:07:28.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:07:28.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:07:28.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:07:28.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:07:28.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:07:28.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5775: TEST_periodic_scrub_replicated: local osd=0 2026-02-17T12:07:28.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5776: TEST_periodic_scrub_replicated: add_something td/osd-scrub-repair psr_pool POBJ scrub 2026-02-17T12:07:28.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T12:07:28.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=psr_pool 2026-02-17T12:07:28.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=POBJ 2026-02-17T12:07:28.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=scrub 2026-02-17T12:07:28.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' scrub = noscrub ']' 2026-02-17T12:07:28.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:81: add_something: ceph osd unset noscrub 2026-02-17T12:07:28.750 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is unset 2026-02-17T12:07:28.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:82: add_something: ceph osd unset nodeep-scrub 2026-02-17T12:07:28.956 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is unset 2026-02-17T12:07:28.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T12:07:28.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T12:07:28.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool psr_pool put POBJ td/osd-scrub-repair/ORIGINAL 2026-02-17T12:07:28.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5777: TEST_periodic_scrub_replicated: get_primary psr_pool POBJ 2026-02-17T12:07:28.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=psr_pool 2026-02-17T12:07:28.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=POBJ 2026-02-17T12:07:28.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map psr_pool POBJ 2026-02-17T12:07:28.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:07:29.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5777: TEST_periodic_scrub_replicated: local primary=0 2026-02-17T12:07:29.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5778: TEST_periodic_scrub_replicated: get_pg psr_pool POBJ 2026-02-17T12:07:29.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=psr_pool 2026-02-17T12:07:29.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=POBJ 2026-02-17T12:07:29.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map psr_pool POBJ 2026-02-17T12:07:29.149 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T12:07:29.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5778: TEST_periodic_scrub_replicated: local pg=2.0 2026-02-17T12:07:29.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5781: TEST_periodic_scrub_replicated: local payload=UVWXYZ 2026-02-17T12:07:29.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5782: TEST_periodic_scrub_replicated: echo UVWXYZ 2026-02-17T12:07:29.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5784: TEST_periodic_scrub_replicated: objectstore_tool td/osd-scrub-repair 0 POBJ set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T12:07:29.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:07:29.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:07:29.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T12:07:29.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:07:29.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 POBJ set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T12:07:29.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:07:29.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:07:29.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T12:07:29.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:07:29.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T12:07:29.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:07:29.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:07:29.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:07:29.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:07:29.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:07:30.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:07:30.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 POBJ set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T12:07:30.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:07:30.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:07:30.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:07:30.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:07:30.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:07:30.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 POBJ set-bytes td/osd-scrub-repair/CORRUPT 2026-02-17T12:07:31.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-17T12:07:31.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:07:31.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:07:31.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:07:31.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:07:31.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:07:31.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:07:31.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:07:31.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:07:31.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:07:31.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:07:31.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:07:31.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:07:31.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:07:31.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:07:31.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:07:31.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:07:31.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:07:31.796 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:07:31.797 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:07:31.797 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:07:31.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:07:31.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:07:31.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:07:31.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:07:31.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:07:31.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:07:31.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:07:31.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:07:31.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:07:31.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:07:31.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0' 2026-02-17T12:07:31.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:07:31.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:07:31.798 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T12:07:31.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-17T12:07:31.798 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T12:07:31.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:07:31.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:07:31.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:07:31.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:07:31.819 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:31.810+0000 7fa889c848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:07:31.821 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:31.810+0000 7fa889c848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:07:31.822 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:31.814+0000 7fa889c848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:07:31.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:07:31.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:07:31.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:07:31.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:07:31.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:07:31.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:31.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:07:31.948 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:07:31.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:31.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:07:32.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:07:33.076 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:33.066+0000 7fa889c848c0 -1 Falling back to public interface 2026-02-17T12:07:33.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:07:33.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:33.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:07:33.105 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:07:33.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:33.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:07:33.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:07:34.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:07:34.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:34.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:07:34.259 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:07:34.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:34.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:07:34.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:07:35.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:07:35.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:35.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:07:35.414 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:07:35.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:35.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:07:35.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:07:36.116 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:07:36.106+0000 7fa889c848c0 -1 osd.0 21 log_to_monitors true 2026-02-17T12:07:36.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:07:36.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:07:36.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:07:36.572 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:07:36.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:07:36.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:07:36.727 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 25 up_thru 25 down_at 22 last_clean_interval [5,21) [v2:127.0.0.1:6802/4200603144,v1:127.0.0.1:6803/4200603144] [v2:127.0.0.1:6804/4200603144,v1:127.0.0.1:6805/4200603144] exists,up 3f07edf3-015f-4490-9564-1f69933cc802 2026-02-17T12:07:36.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:07:36.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:07:36.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:07:36.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:07:36.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:07:36.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:07:36.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:07:36.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:07:36.729 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:07:36.729 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:07:36.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:07:36.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:07:36.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:07:36.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:07:36.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:07:36.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:07:36.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:07:36.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:07:36.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:07:36.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:07:37.000 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:07:37.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:07:37.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:37.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:07:37.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182402 2026-02-17T12:07:37.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182402 2026-02-17T12:07:37.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182402' 2026-02-17T12:07:37.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:37.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:07:37.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705671 2026-02-17T12:07:37.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705671 2026-02-17T12:07:37.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182402 1-38654705671' 2026-02-17T12:07:37.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:37.144 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-107374182402 2026-02-17T12:07:37.144 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:37.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:07:37.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-107374182402 2026-02-17T12:07:37.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:37.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182402 2026-02-17T12:07:37.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 107374182402' 2026-02-17T12:07:37.149 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 107374182402 2026-02-17T12:07:37.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:37.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 107374182402 2026-02-17T12:07:37.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:07:38.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:07:38.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:38.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 107374182402 2026-02-17T12:07:38.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:07:39.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:07:39.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:39.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182402 -lt 107374182402 2026-02-17T12:07:39.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:39.622 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705671 2026-02-17T12:07:39.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:39.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:07:39.625 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705671 2026-02-17T12:07:39.625 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:39.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705671 2026-02-17T12:07:39.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-17T12:07:39.627 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 38654705671 2026-02-17T12:07:39.627 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:07:39.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705671 -lt 38654705671 2026-02-17T12:07:39.784 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:07:39.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:07:39.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:07:39.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:07:39.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:07:39.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:07:39.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:07:39.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:07:39.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:07:39.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:07:39.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:07:40.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:07:40.121 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:07:40.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:07:40.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:07:40.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:07:40.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:07:40.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:07:40.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5787: TEST_periodic_scrub_replicated: set -o pipefail 2026-02-17T12:07:40.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5788: TEST_periodic_scrub_replicated: rados list-inconsistent-obj 2.0 2026-02-17T12:07:40.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5788: TEST_periodic_scrub_replicated: jq . 2026-02-17T12:07:40.320 INFO:tasks.workunit.client.0.trial128.stderr:No scrub information available for pg 2.0 2026-02-17T12:07:40.321 INFO:tasks.workunit.client.0.trial128.stderr:error 2: (2) No such file or directory 2026-02-17T12:07:40.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5789: TEST_periodic_scrub_replicated: set +o pipefail 2026-02-17T12:07:40.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5791: TEST_periodic_scrub_replicated: pg_deep_scrub 2.0 2026-02-17T12:07:40.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: pg_deep_scrub: local pgid=2.0 2026-02-17T12:07:40.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: pg_deep_scrub: wait_for_pg_clean 2.0 2026-02-17T12:07:40.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=2.0 2026-02-17T12:07:40.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T12:07:40.324 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:07:40.325 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:07:40.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:07:40.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:07:40.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:07:40.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T12:07:40.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T12:07:40.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T12:07:40.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T12:07:40.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:07:40.635 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:07:40.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:07:40.790 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:07:40.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:07:40.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:40.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:07:40.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182403 2026-02-17T12:07:40.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182403 2026-02-17T12:07:40.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182403' 2026-02-17T12:07:40.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:40.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:07:40.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705673 2026-02-17T12:07:40.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705673 2026-02-17T12:07:40.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182403 1-38654705673' 2026-02-17T12:07:40.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:40.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-107374182403 2026-02-17T12:07:40.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:40.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:07:40.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-107374182403 2026-02-17T12:07:40.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:40.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182403 2026-02-17T12:07:40.941 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 107374182403 2026-02-17T12:07:40.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 107374182403' 2026-02-17T12:07:40.942 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:41.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182403 -lt 107374182403 2026-02-17T12:07:41.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:41.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705673 2026-02-17T12:07:41.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:41.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:07:41.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705673 2026-02-17T12:07:41.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:41.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705673 2026-02-17T12:07:41.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705673' 2026-02-17T12:07:41.109 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705673 2026-02-17T12:07:41.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:07:41.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705673 -lt 38654705673 2026-02-17T12:07:41.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T12:07:41.271 INFO:tasks.workunit.client.0.trial128.stdout:#---------- 2.0 loop 0 2026-02-17T12:07:41.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 2.0 loop 0' 2026-02-17T12:07:41.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 2.0 2026-02-17T12:07:41.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=2.0 2026-02-17T12:07:41.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T12:07:41.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 2.0 query 2026-02-17T12:07:41.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T12:07:41.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T12:07:41.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T12:07:41.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T12:07:41.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T12:07:41.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: get_last_scrub_stamp 2.0 last_deep_scrub_stamp 2026-02-17T12:07:41.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:07:41.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:07:41.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:07:41.347 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_deep_scrub_stamp' 2026-02-17T12:07:41.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: local last_scrub=2026-02-17T12:07:25.084601+0000 2026-02-17T12:07:41.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: pg_deep_scrub: ceph pg deep-scrub 2.0 2026-02-17T12:07:41.642 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0 on osd.0 to deep-scrub 2026-02-17T12:07:41.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: pg_deep_scrub: wait_for_scrub 2.0 2026-02-17T12:07:25.084601+0000 last_deep_scrub_stamp 2026-02-17T12:07:41.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T12:07:41.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:07:25.084601+0000 2026-02-17T12:07:41.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-17T12:07:41.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:07:41.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:07:41.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_deep_scrub_stamp 2026-02-17T12:07:41.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:07:41.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:07:41.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:07:41.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_deep_scrub_stamp' 2026-02-17T12:07:41.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:07:25.084601+0000 '>' 2026-02-17T12:07:25.084601+0000 2026-02-17T12:07:41.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:07:42.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:07:42.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:07:42.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_deep_scrub_stamp 2026-02-17T12:07:42.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:07:42.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:07:42.812 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_deep_scrub_stamp' 2026-02-17T12:07:42.812 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:07:42.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:07:25.084601+0000 '>' 2026-02-17T12:07:25.084601+0000 2026-02-17T12:07:42.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:07:43.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:07:43.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:07:43.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_deep_scrub_stamp 2026-02-17T12:07:43.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:07:43.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:07:43.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:07:43.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_deep_scrub_stamp' 2026-02-17T12:07:44.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:07:25.084601+0000 '>' 2026-02-17T12:07:25.084601+0000 2026-02-17T12:07:44.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:07:45.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:07:45.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:07:45.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_deep_scrub_stamp 2026-02-17T12:07:45.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:07:45.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:07:45.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:07:45.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_deep_scrub_stamp' 2026-02-17T12:07:45.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:07:25.084601+0000 '>' 2026-02-17T12:07:25.084601+0000 2026-02-17T12:07:45.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:07:46.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:07:46.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:07:46.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_deep_scrub_stamp 2026-02-17T12:07:46.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:07:46.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:07:46.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:07:46.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_deep_scrub_stamp' 2026-02-17T12:07:46.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:07:25.084601+0000 '>' 2026-02-17T12:07:25.084601+0000 2026-02-17T12:07:46.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:07:47.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:07:47.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:07:47.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_deep_scrub_stamp 2026-02-17T12:07:47.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:07:47.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:07:47.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_deep_scrub_stamp' 2026-02-17T12:07:47.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:07:47.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:07:42.070686+0000 '>' 2026-02-17T12:07:25.084601+0000 2026-02-17T12:07:47.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:07:47.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5794: TEST_periodic_scrub_replicated: rados list-inconsistent-obj 2.0 2026-02-17T12:07:47.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5794: TEST_periodic_scrub_replicated: jq . 2026-02-17T12:07:47.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5794: TEST_periodic_scrub_replicated: grep -q POBJ 2026-02-17T12:07:47.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5796: TEST_periodic_scrub_replicated: flush_pg_stats 2026-02-17T12:07:47.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:07:47.612 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:07:47.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:07:47.771 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:07:47.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:07:47.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:47.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:07:47.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182406 2026-02-17T12:07:47.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182406 2026-02-17T12:07:47.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182406' 2026-02-17T12:07:47.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:47.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:07:47.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705675 2026-02-17T12:07:47.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705675 2026-02-17T12:07:47.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182406 1-38654705675' 2026-02-17T12:07:47.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:47.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-107374182406 2026-02-17T12:07:47.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:47.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:07:47.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-107374182406 2026-02-17T12:07:47.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:47.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182406 2026-02-17T12:07:47.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 107374182406' 2026-02-17T12:07:47.921 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 107374182406 2026-02-17T12:07:47.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:48.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182405 -lt 107374182406 2026-02-17T12:07:48.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:07:49.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:07:49.087 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:49.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182406 -lt 107374182406 2026-02-17T12:07:49.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:49.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705675 2026-02-17T12:07:49.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:49.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:07:49.241 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705675 2026-02-17T12:07:49.241 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:49.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705675 2026-02-17T12:07:49.243 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705675 2026-02-17T12:07:49.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-17T12:07:49.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:07:49.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705675 2026-02-17T12:07:49.409 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5797: TEST_periodic_scrub_replicated: get_last_scrub_stamp 2.0 2026-02-17T12:07:49.409 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:07:49.409 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:07:49.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:07:49.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:07:49.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5797: TEST_periodic_scrub_replicated: local last_scrub=2026-02-17T12:07:42.070686+0000 2026-02-17T12:07:49.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5799: TEST_periodic_scrub_replicated: ceph tell 2.0 schedule-scrub 2026-02-17T12:07:49.625 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:07:49.625 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:07:49.625 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:07:49.625 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-10T12:06:09.621611+0000" 2026-02-17T12:07:49.626 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:07:49.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5801: TEST_periodic_scrub_replicated: wait_for_scrub 2.0 2026-02-17T12:07:42.070686+0000 2026-02-17T12:07:49.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T12:07:49.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:07:42.070686+0000 2026-02-17T12:07:49.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:07:49.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:07:49.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:07:49.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:07:49.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:07:49.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:07:49.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:07:49.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:07:49.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:07:42.070686+0000 '>' 2026-02-17T12:07:42.070686+0000 2026-02-17T12:07:49.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:07:50.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:07:50.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:07:50.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:07:50.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:07:50.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:07:50.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:07:50.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:07:50.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:07:42.070686+0000 '>' 2026-02-17T12:07:42.070686+0000 2026-02-17T12:07:50.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:07:51.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:07:51.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:07:51.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:07:51.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:07:51.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:07:51.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:07:51.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:07:52.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:07:42.070686+0000 '>' 2026-02-17T12:07:42.070686+0000 2026-02-17T12:07:52.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:07:53.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:07:53.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:07:53.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:07:53.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:07:53.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:07:53.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:07:53.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:07:53.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:07:50.085653+0000 '>' 2026-02-17T12:07:42.070686+0000 2026-02-17T12:07:53.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:07:53.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5804: TEST_periodic_scrub_replicated: grep -q 'Deep scrub errors, upgrading scrub to deep-scrub' td/osd-scrub-repair/osd.0.log 2026-02-17T12:07:53.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5807: TEST_periodic_scrub_replicated: rados list-inconsistent-obj 2.0 2026-02-17T12:07:53.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5807: TEST_periodic_scrub_replicated: jq . 2026-02-17T12:07:53.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5807: TEST_periodic_scrub_replicated: grep -q POBJ 2026-02-17T12:07:53.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5810: TEST_periodic_scrub_replicated: ceph osd set nodeep-scrub 2026-02-17T12:07:53.477 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:07:53.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5812: TEST_periodic_scrub_replicated: ceph tell osd.0 get_latest_osdmap 2026-02-17T12:07:53.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5813: TEST_periodic_scrub_replicated: flush_pg_stats 2026-02-17T12:07:53.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:07:53.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:07:53.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:07:53.728 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:07:53.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:07:53.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:53.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:07:53.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182408 2026-02-17T12:07:53.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182408 2026-02-17T12:07:53.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182408' 2026-02-17T12:07:53.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:07:53.801 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:07:53.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705677 2026-02-17T12:07:53.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705677 2026-02-17T12:07:53.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182408 1-38654705677' 2026-02-17T12:07:53.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:53.873 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-107374182408 2026-02-17T12:07:53.873 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:53.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:07:53.875 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-107374182408 2026-02-17T12:07:53.875 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:53.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182408 2026-02-17T12:07:53.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 107374182408' 2026-02-17T12:07:53.877 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 107374182408 2026-02-17T12:07:53.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:54.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182407 -lt 107374182408 2026-02-17T12:07:54.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:07:55.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:07:55.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:07:55.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182408 -lt 107374182408 2026-02-17T12:07:55.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:07:55.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705677 2026-02-17T12:07:55.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:07:55.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:07:55.198 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705677 2026-02-17T12:07:55.198 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:07:55.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705677 2026-02-17T12:07:55.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705677' 2026-02-17T12:07:55.200 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705677 2026-02-17T12:07:55.201 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:07:55.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705677 -lt 38654705677 2026-02-17T12:07:55.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5814: TEST_periodic_scrub_replicated: sleep 5 2026-02-17T12:08:00.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5817: TEST_periodic_scrub_replicated: ceph tell 2.0 schedule-scrub 2026-02-17T12:08:00.423 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:08:00.423 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:08:00.423 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:08:00.423 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-10T12:06:20.418474+0000" 2026-02-17T12:08:00.423 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:08:00.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5820: TEST_periodic_scrub_replicated: local found=false 2026-02-17T12:08:00.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5821: TEST_periodic_scrub_replicated: seq 14 -1 0 2026-02-17T12:08:00.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5821: TEST_periodic_scrub_replicated: for i in $(seq 14 -1 0) 2026-02-17T12:08:00.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5823: TEST_periodic_scrub_replicated: sleep 1 2026-02-17T12:08:01.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5824: TEST_periodic_scrub_replicated: grep -q 'Regular scrub skipped due to deep-scrub errors and nodeep-scrub set' td/osd-scrub-repair/osd.0.log 2026-02-17T12:08:01.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5824: TEST_periodic_scrub_replicated: found=true 2026-02-17T12:08:01.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5824: TEST_periodic_scrub_replicated: break 2026-02-17T12:08:01.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5827: TEST_periodic_scrub_replicated: test true = true 2026-02-17T12:08:01.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5830: TEST_periodic_scrub_replicated: rados list-inconsistent-obj 2.0 2026-02-17T12:08:01.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5830: TEST_periodic_scrub_replicated: jq . 2026-02-17T12:08:01.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5830: TEST_periodic_scrub_replicated: grep -q POBJ 2026-02-17T12:08:01.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5832: TEST_periodic_scrub_replicated: flush_pg_stats 2026-02-17T12:08:01.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:08:01.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:08:01.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:08:01.618 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:08:01.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:08:01.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:08:01.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:08:01.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182411 2026-02-17T12:08:01.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182411 2026-02-17T12:08:01.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182411' 2026-02-17T12:08:01.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:08:01.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:08:01.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705680 2026-02-17T12:08:01.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705680 2026-02-17T12:08:01.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182411 1-38654705680' 2026-02-17T12:08:01.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:08:01.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-107374182411 2026-02-17T12:08:01.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:08:01.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:08:01.767 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-107374182411 2026-02-17T12:08:01.767 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:08:01.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182411 2026-02-17T12:08:01.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 107374182411' 2026-02-17T12:08:01.769 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 107374182411 2026-02-17T12:08:01.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:08:01.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182409 -lt 107374182411 2026-02-17T12:08:01.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:08:02.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:08:02.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:08:03.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182411 -lt 107374182411 2026-02-17T12:08:03.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:08:03.091 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705680 2026-02-17T12:08:03.091 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:08:03.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:08:03.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705680 2026-02-17T12:08:03.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:08:03.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705680 2026-02-17T12:08:03.095 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705680 2026-02-17T12:08:03.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705680' 2026-02-17T12:08:03.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:08:03.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705680 -lt 38654705680 2026-02-17T12:08:03.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5834: TEST_periodic_scrub_replicated: pg_scrub 2.0 2026-02-17T12:08:03.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=2.0 2026-02-17T12:08:03.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 2.0 2026-02-17T12:08:03.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=2.0 2026-02-17T12:08:03.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T12:08:03.255 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:08:03.255 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:08:03.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:08:03.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:08:03.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:08:03.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T12:08:03.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T12:08:03.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T12:08:03.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T12:08:03.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:08:03.554 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:08:03.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:08:03.705 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:08:03.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:08:03.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:08:03.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:08:03.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182412 2026-02-17T12:08:03.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182412 2026-02-17T12:08:03.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182412' 2026-02-17T12:08:03.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:08:03.780 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:08:03.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705681 2026-02-17T12:08:03.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705681 2026-02-17T12:08:03.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182412 1-38654705681' 2026-02-17T12:08:03.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:08:03.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-107374182412 2026-02-17T12:08:03.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:08:03.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:08:03.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-107374182412 2026-02-17T12:08:03.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:08:03.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182412 2026-02-17T12:08:03.857 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 107374182412 2026-02-17T12:08:03.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 107374182412' 2026-02-17T12:08:03.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:08:04.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182411 -lt 107374182412 2026-02-17T12:08:04.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:08:05.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:08:05.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:08:05.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182412 -lt 107374182412 2026-02-17T12:08:05.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:08:05.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705681 2026-02-17T12:08:05.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:08:05.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:08:05.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705681 2026-02-17T12:08:05.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:08:05.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705681 2026-02-17T12:08:05.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705681' 2026-02-17T12:08:05.182 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705681 2026-02-17T12:08:05.183 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:08:05.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705681 -lt 38654705681 2026-02-17T12:08:05.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T12:08:05.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 2.0 loop 0' 2026-02-17T12:08:05.339 INFO:tasks.workunit.client.0.trial128.stdout:#---------- 2.0 loop 0 2026-02-17T12:08:05.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 2.0 2026-02-17T12:08:05.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=2.0 2026-02-17T12:08:05.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T12:08:05.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 2.0 query 2026-02-17T12:08:05.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T12:08:05.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean+inconsistent 2026-02-17T12:08:05.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean+inconsistent == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T12:08:05.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T12:08:05.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T12:08:05.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 2.0 2026-02-17T12:08:05.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:08:05.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:08:05.415 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:08:05.415 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:08:05.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-10T12:06:20.418474+0000 2026-02-17T12:08:05.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 2.0 2026-02-17T12:08:05.710 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 2.0 on osd.0 to scrub 2026-02-17T12:08:05.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 2.0 2026-02-10T12:06:20.418474+0000 2026-02-17T12:08:05.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=2.0 2026-02-17T12:08:05.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-10T12:06:20.418474+0000 2026-02-17T12:08:05.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:08:05.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:08:05.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:08:05.720 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:08:05.720 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:08:05.720 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:08:05.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:08:05.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:08:05.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:06:20.418474+0000 '>' 2026-02-10T12:06:20.418474+0000 2026-02-17T12:08:05.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:08:06.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:08:06.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:08:06.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 2.0 last_scrub_stamp 2026-02-17T12:08:06.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=2.0 2026-02-17T12:08:06.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:08:06.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .last_scrub_stamp' 2026-02-17T12:08:06.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:08:07.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:08:06.076007+0000 '>' 2026-02-10T12:06:20.418474+0000 2026-02-17T12:08:07.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:08:07.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5835: TEST_periodic_scrub_replicated: grep -q 'Regular scrub request, deep-scrub details will be lost' td/osd-scrub-repair/osd.0.log 2026-02-17T12:08:07.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5838: TEST_periodic_scrub_replicated: rados list-inconsistent-obj 2.0 2026-02-17T12:08:07.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5838: TEST_periodic_scrub_replicated: jq . 2026-02-17T12:08:07.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5838: TEST_periodic_scrub_replicated: grep -qv POBJ 2026-02-17T12:08:07.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T12:08:07.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:08:07.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:08:07.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:08:07.060 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:08:07.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:08:07.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:08:07.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:08:07.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:08:07.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:08:07.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:08:07.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:08:07.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:08:07.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:08:07.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:08:07.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:08:07.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:08:07.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:08:07.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:08:07.177 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:08:07.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:08:07.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:08:07.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:08:07.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:08:07.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:07.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:08:07.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:08:07.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:08:07.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:08:07.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T12:08:07.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T12:08:07.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T12:08:07.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T12:08:07.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:08:07.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:08:07.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:08:07.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:08:07.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:08:07.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:08:07.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:08:07.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:08:07.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:08:07.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:08:07.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:08:07.198 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:08:07.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:08:07.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:08:07.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:08:07.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:08:07.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:08:07.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:08:07.203 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:08:07.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:08:07.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:08:07.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:08:07.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:08:07.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:07.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:08:07.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:08:07.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:08:07.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:08:07.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T12:08:07.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:08:07.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:07.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:08:07.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T12:08:07.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:08:07.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:08:07.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:08:07.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T12:08:07.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_repair_stats td/osd-scrub-repair 2026-02-17T12:08:07.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:654: TEST_repair_stats: local dir=td/osd-scrub-repair 2026-02-17T12:08:07.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:655: TEST_repair_stats: local poolname=testpool 2026-02-17T12:08:07.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:656: TEST_repair_stats: local OSDS=2 2026-02-17T12:08:07.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:657: TEST_repair_stats: local OBJS=30 2026-02-17T12:08:07.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:659: TEST_repair_stats: local REPAIRS=20 2026-02-17T12:08:07.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:662: TEST_repair_stats: run_mon td/osd-scrub-repair a 2026-02-17T12:08:07.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T12:08:07.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:08:07.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:08:07.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:08:07.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T12:08:07.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T12:08:07.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:08:07.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:08:07.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:08:07.235 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:08:07.235 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:07.235 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:08:07.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:08:07.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:08:07.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:08:07.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:08:07.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:08:07.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:08:07.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:08:07.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:08:07.264 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:08:07.264 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:08:07.264 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:08:07.265 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:08:07.265 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:07.265 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:08:07.265 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:08:07.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:08:07.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T12:08:07.319 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:08:07.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:08:07.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:08:07.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:08:07.321 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:08:07.321 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:08:07.321 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:08:07.321 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:08:07.321 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:08:07.321 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:07.322 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:08:07.322 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:08:07.322 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:08:07.322 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T12:08:07.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:663: TEST_repair_stats: run_mgr td/osd-scrub-repair x 2026-02-17T12:08:07.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T12:08:07.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:08:07.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:08:07.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:08:07.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T12:08:07.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:08:07.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:08:07.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:08:07.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:08:07.467 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:08:07.467 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:07.467 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:08:07.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:08:07.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:08:07.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:08:07.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:665: TEST_repair_stats: local 'ceph_osd_args=--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T12:08:07.488 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:666: TEST_repair_stats: expr 2 - 1 2026-02-17T12:08:07.490 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:666: TEST_repair_stats: seq 0 1 2026-02-17T12:08:07.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:666: TEST_repair_stats: for id in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:08:07.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:667: TEST_repair_stats: run_osd td/osd-scrub-repair 0 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:08:07.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:08:07.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:08:07.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:08:07.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:08:07.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:08:07.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:08:07.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:08:07.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:08:07.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:08:07.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:08:07.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:08:07.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:08:07.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:08:07.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:08:07.494 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:08:07.494 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:08:07.494 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:08:07.494 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:08:07.494 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:07.494 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:08:07.494 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:08:07.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:08:07.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:08:07.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:08:07.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:08:07.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:08:07.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:08:07.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:08:07.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:08:07.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:08:07.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:08:07.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:08:07.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T12:08:07.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:08:07.497 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:08:07.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=46070f20-a90c-486c-ae2b-af9f9bd173de 2026-02-17T12:08:07.497 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 46070f20-a90c-486c-ae2b-af9f9bd173de 2026-02-17T12:08:07.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 46070f20-a90c-486c-ae2b-af9f9bd173de' 2026-02-17T12:08:07.498 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:08:07.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAnWpRp/IHzHRAATxg3XPxUTQAPNpX5ZNLJwg== 2026-02-17T12:08:07.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAnWpRp/IHzHRAATxg3XPxUTQAPNpX5ZNLJwg=="}' 2026-02-17T12:08:07.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 46070f20-a90c-486c-ae2b-af9f9bd173de -i td/osd-scrub-repair/0/new.json 2026-02-17T12:08:07.600 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:08:07.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T12:08:07.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQAnWpRp/IHzHRAATxg3XPxUTQAPNpX5ZNLJwg== --osd-uuid 46070f20-a90c-486c-ae2b-af9f9bd173de 2026-02-17T12:08:07.625 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:07.617+0000 7f2e852aa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:07.627 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:07.621+0000 7f2e852aa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:07.627 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:07.621+0000 7f2e852aa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:07.628 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:07.621+0000 7f2e852aa8c0 -1 bdev(0x55c2dd107c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:08:07.628 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:07.621+0000 7f2e852aa8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T12:08:10.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T12:08:10.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:08:10.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:08:10.672 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:08:10.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:08:10.872 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:08:10.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:08:10.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:08:10.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:08:10.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:08:10.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:08:10.889 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:10.881+0000 7ff92a2bf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:10.890 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:10.885+0000 7ff92a2bf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:10.891 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:10.885+0000 7ff92a2bf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:11.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:08:11.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:08:11.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:08:11.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:08:11.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:08:11.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:11.028 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:08:11.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:08:11.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:11.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:08:11.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:12.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:12.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:12.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:08:12.179 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:08:12.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:12.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:08:12.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:12.371 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:12.365+0000 7ff92a2bf8c0 -1 Falling back to public interface 2026-02-17T12:08:13.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:13.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:13.330 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:08:13.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:08:13.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:13.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:08:13.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:14.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:14.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:14.481 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:08:14.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:08:14.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:14.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:08:14.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:15.401 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:15.393+0000 7ff92a2bf8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:08:15.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:15.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:15.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:08:15.632 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:08:15.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:15.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:08:15.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:16.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:16.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:16.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:08:16.792 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:08:16.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:16.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:08:16.940 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3342341407,v1:127.0.0.1:6803/3342341407] [v2:127.0.0.1:6804/3342341407,v1:127.0.0.1:6805/3342341407] exists,up 46070f20-a90c-486c-ae2b-af9f9bd173de 2026-02-17T12:08:16.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:08:16.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:08:16.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:08:16.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:666: TEST_repair_stats: for id in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:08:16.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:667: TEST_repair_stats: run_osd td/osd-scrub-repair 1 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:08:16.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:08:16.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:08:16.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:08:16.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:08:16.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:08:16.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:08:16.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:08:16.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:08:16.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:08:16.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:08:16.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:08:16.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:08:16.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:08:16.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:08:16.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:08:16.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:08:16.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:08:16.945 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:08:16.946 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:16.946 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:08:16.946 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:08:16.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:08:16.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:08:16.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:08:16.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:08:16.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:08:16.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:08:16.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:08:16.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:08:16.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:08:16.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:08:16.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:08:16.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T12:08:16.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:08:16.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:08:16.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e3f84b05-bdac-464d-9b19-ec2bc7bac26e 2026-02-17T12:08:16.948 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 e3f84b05-bdac-464d-9b19-ec2bc7bac26e 2026-02-17T12:08:16.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 e3f84b05-bdac-464d-9b19-ec2bc7bac26e' 2026-02-17T12:08:16.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:08:16.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAwWpRpk7HLOBAAmycCMV4vFbAwUpEAOnkLEA== 2026-02-17T12:08:16.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAwWpRpk7HLOBAAmycCMV4vFbAwUpEAOnkLEA=="}' 2026-02-17T12:08:16.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e3f84b05-bdac-464d-9b19-ec2bc7bac26e -i td/osd-scrub-repair/1/new.json 2026-02-17T12:08:17.101 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:08:17.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T12:08:17.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQAwWpRpk7HLOBAAmycCMV4vFbAwUpEAOnkLEA== --osd-uuid e3f84b05-bdac-464d-9b19-ec2bc7bac26e 2026-02-17T12:08:17.127 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:17.121+0000 7f177ec318c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:17.128 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:17.121+0000 7f177ec318c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:17.129 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:17.121+0000 7f177ec318c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:17.130 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:17.125+0000 7f177ec318c0 -1 bdev(0x563055c77c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:08:17.130 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:17.125+0000 7f177ec318c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T12:08:19.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T12:08:19.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:08:19.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:08:19.420 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:08:19.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:08:19.614 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:08:19.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:08:19.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:08:19.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:08:19.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:08:19.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:08:19.638 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:19.633+0000 7f21559ad8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:19.640 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:19.633+0000 7f21559ad8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:19.642 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:19.633+0000 7f21559ad8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:19.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:08:19.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:08:19.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:08:19.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:08:19.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:08:19.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:19.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:08:19.771 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:08:19.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:19.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:08:19.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:20.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:20.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:20.924 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:08:20.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:08:20.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:20.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:08:21.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:21.356 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:21.349+0000 7f21559ad8c0 -1 Falling back to public interface 2026-02-17T12:08:22.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:22.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:22.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:08:22.076 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:08:22.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:22.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:08:22.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:23.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:23.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:23.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:08:23.228 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:08:23.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:23.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:08:23.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:24.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:24.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:24.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:08:24.375 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:08:24.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:24.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:08:24.386 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:24.377+0000 7f21559ad8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:08:24.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:25.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:25.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:25.533 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:08:25.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:08:25.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:25.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:08:25.681 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/154466954,v1:127.0.0.1:6811/154466954] [v2:127.0.0.1:6812/154466954,v1:127.0.0.1:6813/154466954] exists,up e3f84b05-bdac-464d-9b19-ec2bc7bac26e 2026-02-17T12:08:25.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:08:25.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:08:25.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:08:25.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:670: TEST_repair_stats: create_pool testpool 1 1 2026-02-17T12:08:25.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create testpool 1 1 2026-02-17T12:08:25.926 INFO:tasks.workunit.client.0.trial128.stderr:pool 'testpool' created 2026-02-17T12:08:25.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:08:26.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:671: TEST_repair_stats: ceph osd pool set testpool size 2 2026-02-17T12:08:27.133 INFO:tasks.workunit.client.0.trial128.stderr:set pool 1 size to 2 2026-02-17T12:08:27.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:672: TEST_repair_stats: wait_for_clean 2026-02-17T12:08:27.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:08:27.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:08:27.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:08:27.146 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:08:27.146 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:08:27.147 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:08:27.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:08:27.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:08:27.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:08:27.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:08:27.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:08:27.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:08:27.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:08:27.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:08:27.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:08:27.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:08:27.428 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:08:27.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:08:27.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:08:27.429 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:08:27.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T12:08:27.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T12:08:27.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T12:08:27.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:08:27.500 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:08:27.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705666 2026-02-17T12:08:27.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705666 2026-02-17T12:08:27.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-38654705666' 2026-02-17T12:08:27.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:08:27.573 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T12:08:27.573 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:08:27.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:08:27.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T12:08:27.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:08:27.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T12:08:27.577 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836484 2026-02-17T12:08:27.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T12:08:27.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:08:27.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T12:08:27.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:08:28.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:08:28.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:08:28.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T12:08:28.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:08:28.892 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705666 2026-02-17T12:08:28.892 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:08:28.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:08:28.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705666 2026-02-17T12:08:28.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:08:28.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705666 2026-02-17T12:08:28.897 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705666 2026-02-17T12:08:28.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705666' 2026-02-17T12:08:28.898 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:08:29.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705666 -lt 38654705666 2026-02-17T12:08:29.058 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:08:29.058 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:08:29.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:08:29.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:08:29.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:08:29.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:08:29.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:08:29.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:08:29.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:08:29.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:08:29.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:08:29.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:08:29.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:08:29.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:08:29.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:08:29.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:08:29.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:08:29.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:08:29.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:675: TEST_repair_stats: local payload=ABCDEF 2026-02-17T12:08:29.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:676: TEST_repair_stats: echo ABCDEF 2026-02-17T12:08:29.573 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: seq 1 30 2026-02-17T12:08:29.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj1 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj2 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj3 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj4 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj5 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj6 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj7 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj8 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj9 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj10 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj11 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj12 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj13 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj14 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj15 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj16 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj17 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj18 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj19 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:29.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:29.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj20 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:30.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:30.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj21 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:30.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:30.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj22 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:30.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:30.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj23 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:30.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:30.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj24 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:30.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:30.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj25 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:30.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:30.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj26 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:30.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:30.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj27 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:30.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:30.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj28 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:30.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:30.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj29 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:30.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:677: TEST_repair_stats: for i in $(seq 1 $OBJS) 2026-02-17T12:08:30.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:679: TEST_repair_stats: rados --pool testpool put obj30 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:08:30.266 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:684: TEST_repair_stats: get_not_primary testpool obj1 2026-02-17T12:08:30.266 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T12:08:30.266 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=obj1 2026-02-17T12:08:30.267 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool obj1 2026-02-17T12:08:30.267 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T12:08:30.267 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T12:08:30.267 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool obj1 2026-02-17T12:08:30.267 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:08:30.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T12:08:30.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool obj1 2026-02-17T12:08:30.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T12:08:30.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:684: TEST_repair_stats: local other=0 2026-02-17T12:08:30.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:685: TEST_repair_stats: get_pg testpool obj1 2026-02-17T12:08:30.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=testpool 2026-02-17T12:08:30.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=obj1 2026-02-17T12:08:30.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map testpool obj1 2026-02-17T12:08:30.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T12:08:30.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:685: TEST_repair_stats: local pgid=1.0 2026-02-17T12:08:30.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:686: TEST_repair_stats: get_primary testpool obj1 2026-02-17T12:08:30.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T12:08:30.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T12:08:30.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool obj1 2026-02-17T12:08:30.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:08:30.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:686: TEST_repair_stats: local primary=1 2026-02-17T12:08:30.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:688: TEST_repair_stats: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T12:08:30.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:08:30.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:08:30.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:08:30.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:08:30.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:08:32.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:08:32.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:689: TEST_repair_stats: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:08:32.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:08:32.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:08:32.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:08:32.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:08:32.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:08:33.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:08:33.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: seq 1 20 2026-02-17T12:08:33.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:33.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 1 % 2 2026-02-17T12:08:33.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T12:08:33.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj1 remove 2026-02-17T12:08:33.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:33.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:33.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:08:33.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:33.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:08:33.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj1 remove 2026-02-17T12:08:34.168 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-17T12:08:34.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:34.705 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 2 % 2 2026-02-17T12:08:34.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T12:08:34.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj2 remove 2026-02-17T12:08:34.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:34.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:34.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:08:34.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:34.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:08:34.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj2 remove 2026-02-17T12:08:35.372 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:104778fc:::obj2:head# 2026-02-17T12:08:35.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:35.924 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 3 % 2 2026-02-17T12:08:35.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T12:08:35.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj3 remove 2026-02-17T12:08:35.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:35.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:35.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:08:35.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:35.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:08:35.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj3 remove 2026-02-17T12:08:36.555 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:8dd16f86:::obj3:head# 2026-02-17T12:08:37.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:37.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 4 % 2 2026-02-17T12:08:37.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T12:08:37.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj4 remove 2026-02-17T12:08:37.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:37.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:37.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:08:37.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:37.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:08:37.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj4 remove 2026-02-17T12:08:37.758 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:0ee9ae15:::obj4:head# 2026-02-17T12:08:38.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:38.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 5 % 2 2026-02-17T12:08:38.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T12:08:38.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj5 remove 2026-02-17T12:08:38.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:38.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:38.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:08:38.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:38.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:08:38.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj5 remove 2026-02-17T12:08:38.922 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:c52c9666:::obj5:head# 2026-02-17T12:08:39.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:39.460 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 6 % 2 2026-02-17T12:08:39.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T12:08:39.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj6 remove 2026-02-17T12:08:39.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:39.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:39.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:08:39.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:39.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:08:39.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj6 remove 2026-02-17T12:08:40.117 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:233a42c1:::obj6:head# 2026-02-17T12:08:40.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:40.653 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 7 % 2 2026-02-17T12:08:40.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T12:08:40.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj7 remove 2026-02-17T12:08:40.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:40.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:40.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:08:40.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:40.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:08:40.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj7 remove 2026-02-17T12:08:41.291 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:9a09113a:::obj7:head# 2026-02-17T12:08:41.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:41.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 8 % 2 2026-02-17T12:08:41.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T12:08:41.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj8 remove 2026-02-17T12:08:41.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:41.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:41.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:08:41.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:41.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:08:41.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj8 remove 2026-02-17T12:08:42.494 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:a557efb1:::obj8:head# 2026-02-17T12:08:43.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:43.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 9 % 2 2026-02-17T12:08:43.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T12:08:43.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj9 remove 2026-02-17T12:08:43.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:43.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:43.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:08:43.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:43.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:08:43.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj9 remove 2026-02-17T12:08:43.663 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:fe351e27:::obj9:head# 2026-02-17T12:08:44.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:44.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 10 % 2 2026-02-17T12:08:44.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T12:08:44.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj10 remove 2026-02-17T12:08:44.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:44.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:44.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:08:44.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:44.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:08:44.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj10 remove 2026-02-17T12:08:44.845 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:d1337354:::obj10:head# 2026-02-17T12:08:45.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:45.380 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 11 % 2 2026-02-17T12:08:45.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T12:08:45.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj11 remove 2026-02-17T12:08:45.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:45.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:45.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:08:45.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:45.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:08:45.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj11 remove 2026-02-17T12:08:46.030 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:eeae2a94:::obj11:head# 2026-02-17T12:08:46.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:46.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 12 % 2 2026-02-17T12:08:46.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T12:08:46.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj12 remove 2026-02-17T12:08:46.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:46.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:46.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:08:46.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:46.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:08:46.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj12 remove 2026-02-17T12:08:47.214 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:af5f95cb:::obj12:head# 2026-02-17T12:08:47.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:47.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 13 % 2 2026-02-17T12:08:47.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T12:08:47.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj13 remove 2026-02-17T12:08:47.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:47.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:47.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:08:47.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:47.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:08:47.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj13 remove 2026-02-17T12:08:48.374 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:a61ad63e:::obj13:head# 2026-02-17T12:08:48.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:48.908 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 14 % 2 2026-02-17T12:08:48.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T12:08:48.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj14 remove 2026-02-17T12:08:48.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:48.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:48.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:08:48.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:48.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:08:48.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj14 remove 2026-02-17T12:08:49.553 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:e0a44829:::obj14:head# 2026-02-17T12:08:50.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:50.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 15 % 2 2026-02-17T12:08:50.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T12:08:50.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj15 remove 2026-02-17T12:08:50.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:50.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:50.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:08:50.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:50.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:08:50.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj15 remove 2026-02-17T12:08:50.731 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:ab946124:::obj15:head# 2026-02-17T12:08:51.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:51.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 16 % 2 2026-02-17T12:08:51.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T12:08:51.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj16 remove 2026-02-17T12:08:51.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:51.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:51.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:08:51.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:51.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:08:51.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj16 remove 2026-02-17T12:08:51.907 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:461f8b5e:::obj16:head# 2026-02-17T12:08:52.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:52.444 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 17 % 2 2026-02-17T12:08:52.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T12:08:52.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj17 remove 2026-02-17T12:08:52.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:52.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:52.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:08:52.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:52.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:08:52.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj17 remove 2026-02-17T12:08:53.072 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:24fc6e92:::obj17:head# 2026-02-17T12:08:53.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:53.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 18 % 2 2026-02-17T12:08:53.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T12:08:53.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj18 remove 2026-02-17T12:08:53.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:53.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:53.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:08:53.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:53.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:08:53.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj18 remove 2026-02-17T12:08:54.250 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:b9836a99:::obj18:head# 2026-02-17T12:08:54.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:54.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 19 % 2 2026-02-17T12:08:54.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=1 2026-02-17T12:08:54.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 1 obj19 remove 2026-02-17T12:08:54.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:54.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:54.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:08:54.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:54.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:08:54.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj19 remove 2026-02-17T12:08:55.423 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:00b31b6c:::obj19:head# 2026-02-17T12:08:55.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:690: TEST_repair_stats: for i in $(seq 1 $REPAIRS) 2026-02-17T12:08:55.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: expr 20 % 2 2026-02-17T12:08:55.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:693: TEST_repair_stats: OSD=0 2026-02-17T12:08:55.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:694: TEST_repair_stats: _objectstore_tool_nodown td/osd-scrub-repair 0 obj20 remove 2026-02-17T12:08:55.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:08:55.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:08:55.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:08:55.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:08:55.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:08:55.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj20 remove 2026-02-17T12:08:56.602 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:24b5611b:::obj20:head# 2026-02-17T12:08:57.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:696: TEST_repair_stats: activate_osd td/osd-scrub-repair 1 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:08:57.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:08:57.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:08:57.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:08:57.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:08:57.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:08:57.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:08:57.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:08:57.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:08:57.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:08:57.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:08:57.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:08:57.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:08:57.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:08:57.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:08:57.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:08:57.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:08:57.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:08:57.137 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:08:57.138 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:08:57.138 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:08:57.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:08:57.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:08:57.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:08:57.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:08:57.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:08:57.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:08:57.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:08:57.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:08:57.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:08:57.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:08:57.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T12:08:57.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:08:57.140 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:08:57.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:08:57.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:08:57.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:08:57.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:08:57.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:08:57.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:08:57.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:08:57.158 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:57.152+0000 7fcc716258c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:57.160 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:57.152+0000 7fcc716258c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:57.161 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:57.152+0000 7fcc716258c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:08:57.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:08:57.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:08:57.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:08:57.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:08:57.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:08:57.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:57.303 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:08:57.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:08:57.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:57.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:08:57.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:58.420 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:08:58.412+0000 7fcc716258c0 -1 Falling back to public interface 2026-02-17T12:08:58.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:58.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:58.459 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:08:58.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:08:58.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:58.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:08:58.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:08:59.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:08:59.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:08:59.609 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:08:59.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:08:59.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:08:59.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:08:59.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:00.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:00.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:00.761 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:09:00.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:09:00.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:00.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:09:00.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:01.407 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:01.400+0000 7fcc716258c0 -1 osd.1 16 log_to_monitors true 2026-02-17T12:09:01.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:01.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:01.918 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:09:01.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:09:01.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:01.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:09:02.067 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 19 up_thru 19 down_at 17 last_clean_interval [9,16) [v2:127.0.0.1:6802/3124537196,v1:127.0.0.1:6803/3124537196] [v2:127.0.0.1:6804/3124537196,v1:127.0.0.1:6805/3124537196] exists,up e3f84b05-bdac-464d-9b19-ec2bc7bac26e 2026-02-17T12:09:02.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:09:02.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:09:02.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:09:02.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:697: TEST_repair_stats: activate_osd td/osd-scrub-repair 0 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:09:02.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:09:02.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:09:02.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:09:02.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:09:02.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:09:02.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:09:02.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:09:02.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:09:02.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:09:02.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:09:02.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:09:02.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:09:02.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:09:02.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:09:02.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:09:02.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:09:02.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:09:02.072 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:09:02.072 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:02.072 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:09:02.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:09:02.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:09:02.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:09:02.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:09:02.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:09:02.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:09:02.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:09:02.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:09:02.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:09:02.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:09:02.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T12:09:02.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:09:02.074 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:09:02.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:09:02.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:09:02.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T12:09:02.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:09:02.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:09:02.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:09:02.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:09:02.090 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:02.084+0000 7fe8e19338c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:02.091 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:02.084+0000 7fe8e19338c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:02.093 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:02.084+0000 7fe8e19338c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:02.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:09:02.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:09:02.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:09:02.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:09:02.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:09:02.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:02.231 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:09:02.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:09:02.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:02.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:09:02.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:03.360 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:03.352+0000 7fe8e19338c0 -1 Falling back to public interface 2026-02-17T12:09:03.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:03.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:03.382 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:09:03.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:09:03.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:03.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:09:03.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:04.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:04.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:04.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:09:04.526 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:09:04.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:04.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:09:04.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:05.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:05.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:05.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:09:05.676 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:09:05.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:05.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:09:05.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:06.306 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:06.300+0000 7fe8e19338c0 -1 osd.0 14 log_to_monitors true 2026-02-17T12:09:06.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:06.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:06.826 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:09:06.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:09:06.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:06.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:09:06.977 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 23 up_thru 0 down_at 15 last_clean_interval [5,14) [v2:127.0.0.1:6810/636521913,v1:127.0.0.1:6811/636521913] [v2:127.0.0.1:6812/636521913,v1:127.0.0.1:6813/636521913] exists,up 46070f20-a90c-486c-ae2b-af9f9bd173de 2026-02-17T12:09:06.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:09:06.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:09:06.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:09:06.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:698: TEST_repair_stats: wait_for_clean 2026-02-17T12:09:06.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:09:06.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:09:06.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:09:06.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:09:06.979 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:09:06.979 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:09:06.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:09:06.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:09:06.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:09:07.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:09:07.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:09:07.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:09:07.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:09:07.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:09:07.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:09:07.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:09:07.254 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:09:07.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:09:07.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:09:07.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:09:07.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247810 2026-02-17T12:09:07.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247810 2026-02-17T12:09:07.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810' 2026-02-17T12:09:07.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:09:07.329 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:09:07.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=81604378627 2026-02-17T12:09:07.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 81604378627 2026-02-17T12:09:07.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810 1-81604378627' 2026-02-17T12:09:07.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:09:07.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247810 2026-02-17T12:09:07.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:09:07.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:09:07.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247810 2026-02-17T12:09:07.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:09:07.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247810 2026-02-17T12:09:07.405 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 98784247810 2026-02-17T12:09:07.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247810' 2026-02-17T12:09:07.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:09:07.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 98784247810 2026-02-17T12:09:07.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:09:08.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:09:08.572 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:09:08.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247810 -lt 98784247810 2026-02-17T12:09:08.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:09:08.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-81604378627 2026-02-17T12:09:08.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:09:08.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:09:08.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-81604378627 2026-02-17T12:09:08.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:09:08.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=81604378627 2026-02-17T12:09:08.730 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 81604378627 2026-02-17T12:09:08.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 81604378627' 2026-02-17T12:09:08.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:09:08.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 81604378627 -lt 81604378627 2026-02-17T12:09:08.896 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:09:08.896 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:09:08.896 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:09:09.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:09:09.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:09:09.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:09:09.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:09:09.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:09:09.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:09:09.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:09:09.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:09:09.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T12:09:09.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:09:09.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:09:09.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:09:09.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T12:09:09.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-17T12:09:09.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-17T12:09:09.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-17T12:09:09.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T12:09:09.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-17T12:09:09.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T12:09:09.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:09:09.527 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:09:09.527 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:09:09.527 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:09:09.527 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:09:09.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:09:09.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:09:09.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T12:09:09.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:09:09.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:09:09.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:09:09.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T12:09:09.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T12:09:09.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T12:09:09.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T12:09:09.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T12:09:09.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T12:09:09.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T12:09:09.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T12:09:09.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T12:09:10.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T12:09:10.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T12:09:10.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 13 )) 2026-02-17T12:09:10.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T12:09:10.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-17T12:09:10.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T12:09:10.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:09:10.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:09:10.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:09:10.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:09:10.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:09:10.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:09:10.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:09:10.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T12:09:10.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:09:10.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:09:10.409 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:09:10.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T12:09:10.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T12:09:10.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T12:09:10.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T12:09:10.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T12:09:10.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T12:09:10.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T12:09:10.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T12:09:10.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T12:09:10.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T12:09:10.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T12:09:10.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 13 )) 2026-02-17T12:09:10.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T12:09:10.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-17T12:09:11.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T12:09:11.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:09:11.189 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:09:11.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:09:11.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:09:11.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:09:11.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:09:11.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:09:11.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-17T12:09:11.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:09:11.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:09:11.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:09:11.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-17T12:09:11.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-17T12:09:11.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-17T12:09:11.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-17T12:09:11.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-17T12:09:11.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-17T12:09:11.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-17T12:09:11.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-17T12:09:11.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-17T12:09:11.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-17T12:09:11.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-17T12:09:11.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 13 )) 2026-02-17T12:09:11.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-17T12:09:11.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-17T12:09:12.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-17T12:09:12.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:09:12.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:09:12.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:09:12.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:09:12.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:09:12.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:09:12.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:09:12.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:09:12.659 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:09:12.659 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:09:12.659 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:09:12.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:09:12.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:09:12.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:09:12.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:700: TEST_repair_stats: repair 1.0 2026-02-17T12:09:12.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.0 2026-02-17T12:09:12.838 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.0 2026-02-17T12:09:12.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:09:12.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:09:12.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:09:12.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:09:12.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T12:08:25.919695+0000 2026-02-17T12:09:12.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.0 2026-02-17T12:09:13.131 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 1.0 on osd.1 to repair 2026-02-17T12:09:13.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.0 2026-02-17T12:08:25.919695+0000 2026-02-17T12:09:13.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T12:09:13.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:08:25.919695+0000 2026-02-17T12:09:13.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:09:13.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:09:13.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:09:13.141 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:09:13.141 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:09:13.141 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:09:13.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:09:13.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:09:13.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:08:25.919695+0000 '>' 2026-02-17T12:08:25.919695+0000 2026-02-17T12:09:13.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:09:14.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:09:14.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:09:14.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:09:14.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:09:14.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:09:14.294 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:09:14.294 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:09:14.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:08:25.919695+0000 '>' 2026-02-17T12:08:25.919695+0000 2026-02-17T12:09:14.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:09:15.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:09:15.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:09:15.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:09:15.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:09:15.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:09:15.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:09:15.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:09:15.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:08:25.919695+0000 '>' 2026-02-17T12:08:25.919695+0000 2026-02-17T12:09:15.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:09:16.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:09:16.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:09:16.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:09:16.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:09:16.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:09:16.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:09:16.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:09:16.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:08:25.919695+0000 '>' 2026-02-17T12:08:25.919695+0000 2026-02-17T12:09:16.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:09:17.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:09:17.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:09:17.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:09:17.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:09:17.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:09:17.749 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:09:17.749 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:09:17.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:09:13.281350+0000 '>' 2026-02-17T12:08:25.919695+0000 2026-02-17T12:09:17.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:09:17.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:701: TEST_repair_stats: wait_for_clean 2026-02-17T12:09:17.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:09:17.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:09:17.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:09:17.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:09:17.900 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:09:17.900 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:09:17.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:09:17.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:09:17.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:09:18.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:09:18.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:09:18.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:09:18.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:09:18.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:09:18.025 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:09:18.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:09:18.180 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:09:18.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:09:18.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:09:18.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:09:18.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247813 2026-02-17T12:09:18.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247813 2026-02-17T12:09:18.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247813' 2026-02-17T12:09:18.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:09:18.253 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:09:18.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=81604378630 2026-02-17T12:09:18.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 81604378630 2026-02-17T12:09:18.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247813 1-81604378630' 2026-02-17T12:09:18.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:09:18.326 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247813 2026-02-17T12:09:18.326 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:09:18.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:09:18.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247813 2026-02-17T12:09:18.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:09:18.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247813 2026-02-17T12:09:18.330 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 98784247813 2026-02-17T12:09:18.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247813' 2026-02-17T12:09:18.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:09:18.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247812 -lt 98784247813 2026-02-17T12:09:18.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:09:19.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:09:19.493 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:09:19.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247812 -lt 98784247813 2026-02-17T12:09:19.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:09:20.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:09:20.647 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:09:20.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247813 -lt 98784247813 2026-02-17T12:09:20.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:09:20.804 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-81604378630 2026-02-17T12:09:20.805 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:09:20.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:09:20.807 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-81604378630 2026-02-17T12:09:20.807 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:09:20.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=81604378630 2026-02-17T12:09:20.809 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 81604378630 2026-02-17T12:09:20.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 81604378630' 2026-02-17T12:09:20.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:09:20.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 81604378630 -lt 81604378630 2026-02-17T12:09:20.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:09:20.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:09:20.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:09:21.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:09:21.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:09:21.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:09:21.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:09:21.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:09:21.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:09:21.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:09:21.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:09:21.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:09:21.303 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:09:21.303 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:09:21.303 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:09:21.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:09:21.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:09:21.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:09:21.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:702: TEST_repair_stats: ceph pg dump pgs 2026-02-17T12:09:21.628 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T12:09:21.628 INFO:tasks.workunit.client.0.trial128.stdout:1.0 30 0 0 0 0 210 0 0 30 0 30 active+clean 2026-02-17T12:09:13.299100+0000 14'30 25:127 [1,0] 1 [1,0] 1 14'30 2026-02-17T12:09:13.281350+0000 14'30 2026-02-17T12:09:13.281350+0000 0 1 periodic scrub scheduled @ 2026-02-18T12:09:13.281350+0000 20 0 2026-02-17T12:09:21.628 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T12:09:21.628 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T12:09:21.628 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:09:21.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:703: TEST_repair_stats: flush_pg_stats 2026-02-17T12:09:21.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:09:21.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:09:21.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:09:21.809 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:09:21.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:09:21.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:09:21.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:09:21.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247815 2026-02-17T12:09:21.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247815 2026-02-17T12:09:21.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247815' 2026-02-17T12:09:21.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:09:21.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:09:21.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=81604378632 2026-02-17T12:09:21.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 81604378632 2026-02-17T12:09:21.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247815 1-81604378632' 2026-02-17T12:09:21.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:09:21.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247815 2026-02-17T12:09:21.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:09:21.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:09:21.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247815 2026-02-17T12:09:21.956 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:09:21.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247815 2026-02-17T12:09:21.957 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 98784247815 2026-02-17T12:09:21.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247815' 2026-02-17T12:09:21.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:09:22.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247814 -lt 98784247815 2026-02-17T12:09:22.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:09:23.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:09:23.119 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:09:23.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247814 -lt 98784247815 2026-02-17T12:09:23.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:09:24.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:09:24.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:09:24.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247815 -lt 98784247815 2026-02-17T12:09:24.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:09:24.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-81604378632 2026-02-17T12:09:24.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:09:24.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:09:24.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-81604378632 2026-02-17T12:09:24.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:09:24.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=81604378632 2026-02-17T12:09:24.438 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 81604378632 2026-02-17T12:09:24.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 81604378632' 2026-02-17T12:09:24.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:09:24.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 81604378632 -lt 81604378632 2026-02-17T12:09:24.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:706: TEST_repair_stats: ceph pg 1.0 query 2026-02-17T12:09:24.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:706: TEST_repair_stats: jq .info.stats.stat_sum 2026-02-17T12:09:24.674 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:09:24.674 INFO:tasks.workunit.client.0.trial128.stdout: "num_bytes": 210, 2026-02-17T12:09:24.674 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects": 30, 2026-02-17T12:09:24.674 INFO:tasks.workunit.client.0.trial128.stdout: "num_object_clones": 0, 2026-02-17T12:09:24.674 INFO:tasks.workunit.client.0.trial128.stdout: "num_object_copies": 60, 2026-02-17T12:09:24.674 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T12:09:24.675 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_missing": 0, 2026-02-17T12:09:24.675 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_degraded": 0, 2026-02-17T12:09:24.675 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_misplaced": 0, 2026-02-17T12:09:24.675 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_unfound": 0, 2026-02-17T12:09:24.675 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_dirty": 30, 2026-02-17T12:09:24.675 INFO:tasks.workunit.client.0.trial128.stdout: "num_whiteouts": 0, 2026-02-17T12:09:24.675 INFO:tasks.workunit.client.0.trial128.stdout: "num_read": 0, 2026-02-17T12:09:24.675 INFO:tasks.workunit.client.0.trial128.stdout: "num_read_kb": 0, 2026-02-17T12:09:24.675 INFO:tasks.workunit.client.0.trial128.stdout: "num_write": 30, 2026-02-17T12:09:24.675 INFO:tasks.workunit.client.0.trial128.stdout: "num_write_kb": 30, 2026-02-17T12:09:24.675 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrub_errors": 0, 2026-02-17T12:09:24.676 INFO:tasks.workunit.client.0.trial128.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T12:09:24.676 INFO:tasks.workunit.client.0.trial128.stdout: "num_deep_scrub_errors": 0, 2026-02-17T12:09:24.676 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_recovered": 20, 2026-02-17T12:09:24.676 INFO:tasks.workunit.client.0.trial128.stdout: "num_bytes_recovered": 140, 2026-02-17T12:09:24.676 INFO:tasks.workunit.client.0.trial128.stdout: "num_keys_recovered": 0, 2026-02-17T12:09:24.676 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_omap": 0, 2026-02-17T12:09:24.676 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T12:09:24.676 INFO:tasks.workunit.client.0.trial128.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T12:09:24.676 INFO:tasks.workunit.client.0.trial128.stdout: "num_flush": 0, 2026-02-17T12:09:24.676 INFO:tasks.workunit.client.0.trial128.stdout: "num_flush_kb": 0, 2026-02-17T12:09:24.676 INFO:tasks.workunit.client.0.trial128.stdout: "num_evict": 0, 2026-02-17T12:09:24.676 INFO:tasks.workunit.client.0.trial128.stdout: "num_evict_kb": 0, 2026-02-17T12:09:24.677 INFO:tasks.workunit.client.0.trial128.stdout: "num_promote": 0, 2026-02-17T12:09:24.677 INFO:tasks.workunit.client.0.trial128.stdout: "num_flush_mode_high": 0, 2026-02-17T12:09:24.677 INFO:tasks.workunit.client.0.trial128.stdout: "num_flush_mode_low": 0, 2026-02-17T12:09:24.677 INFO:tasks.workunit.client.0.trial128.stdout: "num_evict_mode_some": 0, 2026-02-17T12:09:24.677 INFO:tasks.workunit.client.0.trial128.stdout: "num_evict_mode_full": 0, 2026-02-17T12:09:24.677 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_pinned": 0, 2026-02-17T12:09:24.677 INFO:tasks.workunit.client.0.trial128.stdout: "num_legacy_snapsets": 0, 2026-02-17T12:09:24.677 INFO:tasks.workunit.client.0.trial128.stdout: "num_large_omap_objects": 0, 2026-02-17T12:09:24.677 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_manifest": 0, 2026-02-17T12:09:24.677 INFO:tasks.workunit.client.0.trial128.stdout: "num_omap_bytes": 0, 2026-02-17T12:09:24.677 INFO:tasks.workunit.client.0.trial128.stdout: "num_omap_keys": 0, 2026-02-17T12:09:24.677 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_repaired": 20 2026-02-17T12:09:24.678 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:09:24.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:707: TEST_repair_stats: ceph pg 1.0 query 2026-02-17T12:09:24.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:707: TEST_repair_stats: jq .info.stats.stat_sum.num_objects_repaired 2026-02-17T12:09:24.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:707: TEST_repair_stats: COUNT=20 2026-02-17T12:09:24.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:708: TEST_repair_stats: test 20 = 20 2026-02-17T12:09:24.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:710: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-17T12:09:24.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:710: TEST_repair_stats: jq '.pg_map.osd_stats[] | select(.osd == 1 )' 2026-02-17T12:09:24.890 INFO:tasks.workunit.client.0.trial128.stderr:dumped all 2026-02-17T12:09:24.899 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:09:24.899 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:09:24.899 INFO:tasks.workunit.client.0.trial128.stdout: "up_from": 19, 2026-02-17T12:09:24.899 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 81604378632, 2026-02-17T12:09:24.899 INFO:tasks.workunit.client.0.trial128.stdout: "num_pgs": 1, 2026-02-17T12:09:24.899 INFO:tasks.workunit.client.0.trial128.stdout: "num_osds": 1, 2026-02-17T12:09:24.899 INFO:tasks.workunit.client.0.trial128.stdout: "num_per_pool_osds": 1, 2026-02-17T12:09:24.900 INFO:tasks.workunit.client.0.trial128.stdout: "num_per_pool_omap_osds": 1, 2026-02-17T12:09:24.900 INFO:tasks.workunit.client.0.trial128.stdout: "kb": 104857600, 2026-02-17T12:09:24.900 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used": 28568, 2026-02-17T12:09:24.900 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_data": 328, 2026-02-17T12:09:24.900 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_omap": 4, 2026-02-17T12:09:24.900 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_meta": 28219, 2026-02-17T12:09:24.900 INFO:tasks.workunit.client.0.trial128.stdout: "kb_avail": 104829032, 2026-02-17T12:09:24.900 INFO:tasks.workunit.client.0.trial128.stdout: "statfs": { 2026-02-17T12:09:24.900 INFO:tasks.workunit.client.0.trial128.stdout: "total": 107374182400, 2026-02-17T12:09:24.900 INFO:tasks.workunit.client.0.trial128.stdout: "available": 107344928768, 2026-02-17T12:09:24.900 INFO:tasks.workunit.client.0.trial128.stdout: "internally_reserved": 0, 2026-02-17T12:09:24.900 INFO:tasks.workunit.client.0.trial128.stdout: "allocated": 335872, 2026-02-17T12:09:24.900 INFO:tasks.workunit.client.0.trial128.stdout: "data_stored": 56968, 2026-02-17T12:09:24.900 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed": 0, 2026-02-17T12:09:24.900 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed_allocated": 0, 2026-02-17T12:09:24.901 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed_original": 0, 2026-02-17T12:09:24.901 INFO:tasks.workunit.client.0.trial128.stdout: "omap_allocated": 4397, 2026-02-17T12:09:24.901 INFO:tasks.workunit.client.0.trial128.stdout: "internal_metadata": 28896979 2026-02-17T12:09:24.901 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:09:24.901 INFO:tasks.workunit.client.0.trial128.stdout: "hb_peers": [ 2026-02-17T12:09:24.901 INFO:tasks.workunit.client.0.trial128.stdout: 0 2026-02-17T12:09:24.901 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:09:24.901 INFO:tasks.workunit.client.0.trial128.stdout: "snap_trim_queue_len": 0, 2026-02-17T12:09:24.901 INFO:tasks.workunit.client.0.trial128.stdout: "num_snap_trimming": 0, 2026-02-17T12:09:24.901 INFO:tasks.workunit.client.0.trial128.stdout: "num_shards_repaired": 10, 2026-02-17T12:09:24.901 INFO:tasks.workunit.client.0.trial128.stdout: "op_queue_age_hist": { 2026-02-17T12:09:24.901 INFO:tasks.workunit.client.0.trial128.stdout: "histogram": [], 2026-02-17T12:09:24.901 INFO:tasks.workunit.client.0.trial128.stdout: "upper_bound": 1 2026-02-17T12:09:24.901 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:09:24.901 INFO:tasks.workunit.client.0.trial128.stdout: "perf_stat": { 2026-02-17T12:09:24.902 INFO:tasks.workunit.client.0.trial128.stdout: "commit_latency_ms": 0, 2026-02-17T12:09:24.902 INFO:tasks.workunit.client.0.trial128.stdout: "apply_latency_ms": 0, 2026-02-17T12:09:24.902 INFO:tasks.workunit.client.0.trial128.stdout: "commit_latency_ns": 0, 2026-02-17T12:09:24.902 INFO:tasks.workunit.client.0.trial128.stdout: "apply_latency_ns": 0 2026-02-17T12:09:24.902 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:09:24.902 INFO:tasks.workunit.client.0.trial128.stdout: "alerts": [] 2026-02-17T12:09:24.902 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:09:24.902 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:711: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-17T12:09:24.903 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:711: TEST_repair_stats: jq '.pg_map.osd_stats[] | select(.osd == 1 ).num_shards_repaired' 2026-02-17T12:09:25.045 INFO:tasks.workunit.client.0.trial128.stderr:dumped all 2026-02-17T12:09:25.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:711: TEST_repair_stats: COUNT=10 2026-02-17T12:09:25.054 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:712: TEST_repair_stats: expr 20 / 2 2026-02-17T12:09:25.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:712: TEST_repair_stats: test 10 = 10 2026-02-17T12:09:25.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:714: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-17T12:09:25.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:714: TEST_repair_stats: jq '.pg_map.osd_stats[] | select(.osd == 0 )' 2026-02-17T12:09:25.200 INFO:tasks.workunit.client.0.trial128.stderr:dumped all 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "up_from": 23, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 98784247815, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "num_pgs": 1, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "num_osds": 1, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "num_per_pool_osds": 1, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "num_per_pool_omap_osds": 1, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "kb": 104857600, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used": 28568, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_data": 328, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_omap": 4, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_meta": 28219, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "kb_avail": 104829032, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "statfs": { 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "total": 107374182400, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "available": 107344928768, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "internally_reserved": 0, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "allocated": 335872, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "data_stored": 56968, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed": 0, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed_allocated": 0, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed_original": 0, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "omap_allocated": 4398, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "internal_metadata": 28896978 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "hb_peers": [ 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:09:25.209 INFO:tasks.workunit.client.0.trial128.stdout: "snap_trim_queue_len": 0, 2026-02-17T12:09:25.210 INFO:tasks.workunit.client.0.trial128.stdout: "num_snap_trimming": 0, 2026-02-17T12:09:25.210 INFO:tasks.workunit.client.0.trial128.stdout: "num_shards_repaired": 10, 2026-02-17T12:09:25.210 INFO:tasks.workunit.client.0.trial128.stdout: "op_queue_age_hist": { 2026-02-17T12:09:25.210 INFO:tasks.workunit.client.0.trial128.stdout: "histogram": [], 2026-02-17T12:09:25.210 INFO:tasks.workunit.client.0.trial128.stdout: "upper_bound": 1 2026-02-17T12:09:25.210 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:09:25.210 INFO:tasks.workunit.client.0.trial128.stdout: "perf_stat": { 2026-02-17T12:09:25.210 INFO:tasks.workunit.client.0.trial128.stdout: "commit_latency_ms": 0, 2026-02-17T12:09:25.210 INFO:tasks.workunit.client.0.trial128.stdout: "apply_latency_ms": 0, 2026-02-17T12:09:25.210 INFO:tasks.workunit.client.0.trial128.stdout: "commit_latency_ns": 0, 2026-02-17T12:09:25.210 INFO:tasks.workunit.client.0.trial128.stdout: "apply_latency_ns": 0 2026-02-17T12:09:25.210 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:09:25.210 INFO:tasks.workunit.client.0.trial128.stdout: "alerts": [] 2026-02-17T12:09:25.210 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:09:25.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:715: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-17T12:09:25.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:715: TEST_repair_stats: jq '.pg_map.osd_stats[] | select(.osd == 0 ).num_shards_repaired' 2026-02-17T12:09:25.349 INFO:tasks.workunit.client.0.trial128.stderr:dumped all 2026-02-17T12:09:25.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:715: TEST_repair_stats: COUNT=10 2026-02-17T12:09:25.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:716: TEST_repair_stats: expr 20 / 2 2026-02-17T12:09:25.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:716: TEST_repair_stats: test 10 = 10 2026-02-17T12:09:25.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:718: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-17T12:09:25.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:718: TEST_repair_stats: jq .pg_map.osd_stats_sum 2026-02-17T12:09:25.496 INFO:tasks.workunit.client.0.trial128.stderr:dumped all 2026-02-17T12:09:25.505 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:09:25.505 INFO:tasks.workunit.client.0.trial128.stdout: "up_from": 0, 2026-02-17T12:09:25.505 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 0, 2026-02-17T12:09:25.505 INFO:tasks.workunit.client.0.trial128.stdout: "num_pgs": 2, 2026-02-17T12:09:25.505 INFO:tasks.workunit.client.0.trial128.stdout: "num_osds": 2, 2026-02-17T12:09:25.505 INFO:tasks.workunit.client.0.trial128.stdout: "num_per_pool_osds": 2, 2026-02-17T12:09:25.505 INFO:tasks.workunit.client.0.trial128.stdout: "num_per_pool_omap_osds": 2, 2026-02-17T12:09:25.506 INFO:tasks.workunit.client.0.trial128.stdout: "kb": 209715200, 2026-02-17T12:09:25.506 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used": 57136, 2026-02-17T12:09:25.506 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_data": 656, 2026-02-17T12:09:25.506 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_omap": 8, 2026-02-17T12:09:25.506 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_meta": 56439, 2026-02-17T12:09:25.506 INFO:tasks.workunit.client.0.trial128.stdout: "kb_avail": 209658064, 2026-02-17T12:09:25.506 INFO:tasks.workunit.client.0.trial128.stdout: "statfs": { 2026-02-17T12:09:25.506 INFO:tasks.workunit.client.0.trial128.stdout: "total": 214748364800, 2026-02-17T12:09:25.506 INFO:tasks.workunit.client.0.trial128.stdout: "available": 214689857536, 2026-02-17T12:09:25.506 INFO:tasks.workunit.client.0.trial128.stdout: "internally_reserved": 0, 2026-02-17T12:09:25.506 INFO:tasks.workunit.client.0.trial128.stdout: "allocated": 671744, 2026-02-17T12:09:25.506 INFO:tasks.workunit.client.0.trial128.stdout: "data_stored": 113936, 2026-02-17T12:09:25.507 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed": 0, 2026-02-17T12:09:25.507 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed_allocated": 0, 2026-02-17T12:09:25.507 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed_original": 0, 2026-02-17T12:09:25.507 INFO:tasks.workunit.client.0.trial128.stdout: "omap_allocated": 8795, 2026-02-17T12:09:25.507 INFO:tasks.workunit.client.0.trial128.stdout: "internal_metadata": 57793957 2026-02-17T12:09:25.507 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:09:25.507 INFO:tasks.workunit.client.0.trial128.stdout: "hb_peers": [], 2026-02-17T12:09:25.507 INFO:tasks.workunit.client.0.trial128.stdout: "snap_trim_queue_len": 0, 2026-02-17T12:09:25.507 INFO:tasks.workunit.client.0.trial128.stdout: "num_snap_trimming": 0, 2026-02-17T12:09:25.507 INFO:tasks.workunit.client.0.trial128.stdout: "num_shards_repaired": 20, 2026-02-17T12:09:25.507 INFO:tasks.workunit.client.0.trial128.stdout: "op_queue_age_hist": { 2026-02-17T12:09:25.507 INFO:tasks.workunit.client.0.trial128.stdout: "histogram": [], 2026-02-17T12:09:25.508 INFO:tasks.workunit.client.0.trial128.stdout: "upper_bound": 1 2026-02-17T12:09:25.508 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:09:25.508 INFO:tasks.workunit.client.0.trial128.stdout: "perf_stat": { 2026-02-17T12:09:25.508 INFO:tasks.workunit.client.0.trial128.stdout: "commit_latency_ms": 0, 2026-02-17T12:09:25.508 INFO:tasks.workunit.client.0.trial128.stdout: "apply_latency_ms": 0, 2026-02-17T12:09:25.508 INFO:tasks.workunit.client.0.trial128.stdout: "commit_latency_ns": 0, 2026-02-17T12:09:25.508 INFO:tasks.workunit.client.0.trial128.stdout: "apply_latency_ns": 0 2026-02-17T12:09:25.508 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:09:25.508 INFO:tasks.workunit.client.0.trial128.stdout: "alerts": [], 2026-02-17T12:09:25.508 INFO:tasks.workunit.client.0.trial128.stdout: "network_ping_times": [] 2026-02-17T12:09:25.508 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:09:25.509 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:719: TEST_repair_stats: ceph pg dump --format=json-pretty 2026-02-17T12:09:25.509 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:719: TEST_repair_stats: jq .pg_map.osd_stats_sum.num_shards_repaired 2026-02-17T12:09:25.651 INFO:tasks.workunit.client.0.trial128.stderr:dumped all 2026-02-17T12:09:25.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:719: TEST_repair_stats: COUNT=20 2026-02-17T12:09:25.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:720: TEST_repair_stats: test 20 = 20 2026-02-17T12:09:25.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T12:09:25.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:09:25.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:09:25.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:09:25.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:09:25.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:09:25.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:09:25.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:09:25.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:09:25.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:09:25.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:09:25.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:09:25.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:09:25.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:09:25.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:09:25.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:09:25.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:09:25.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:09:25.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:09:25.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:09:25.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:09:25.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:09:25.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:09:25.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:09:25.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:25.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:09:25.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:09:25.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:09:25.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:09:25.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T12:09:25.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T12:09:25.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T12:09:25.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T12:09:25.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:09:25.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:09:25.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:09:25.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:09:25.800 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:09:25.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:09:25.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:09:25.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:09:25.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:09:25.801 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:09:25.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:09:25.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:09:25.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:09:25.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:09:25.804 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:09:25.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:09:25.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:09:25.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:09:25.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:09:25.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:09:25.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:09:25.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:09:25.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:09:25.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:25.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:09:25.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:09:25.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:09:25.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:09:25.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T12:09:25.813 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:09:25.813 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:25.813 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:09:25.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T12:09:25.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:09:25.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:09:25.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:09:25.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T12:09:25.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_repair_stats_ec td/osd-scrub-repair 2026-02-17T12:09:25.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:724: TEST_repair_stats_ec: local dir=td/osd-scrub-repair 2026-02-17T12:09:25.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:725: TEST_repair_stats_ec: local poolname=testpool 2026-02-17T12:09:25.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:726: TEST_repair_stats_ec: local OSDS=3 2026-02-17T12:09:25.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:727: TEST_repair_stats_ec: local OBJS=30 2026-02-17T12:09:25.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:729: TEST_repair_stats_ec: local REPAIRS=26 2026-02-17T12:09:25.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:730: TEST_repair_stats_ec: local allow_overwrites=false 2026-02-17T12:09:25.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:733: TEST_repair_stats_ec: run_mon td/osd-scrub-repair a 2026-02-17T12:09:25.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T12:09:25.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:09:25.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:09:25.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:09:25.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T12:09:25.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T12:09:25.837 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:09:25.837 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:09:25.837 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:09:25.838 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:09:25.838 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:25.838 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:09:25.838 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:09:25.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:09:25.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:09:25.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:09:25.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:09:25.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:09:25.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:09:25.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:09:25.866 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:09:25.867 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:09:25.867 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:09:25.867 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:09:25.867 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:25.867 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:09:25.867 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:09:25.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:09:25.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T12:09:25.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:09:25.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:09:25.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:09:25.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:09:25.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:09:25.928 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:09:25.928 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:09:25.928 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:09:25.928 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:09:25.928 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:25.928 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:09:25.928 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:09:25.928 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:09:25.928 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T12:09:25.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:734: TEST_repair_stats_ec: run_mgr td/osd-scrub-repair x 2026-02-17T12:09:25.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T12:09:25.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:09:25.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:09:25.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:09:25.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T12:09:25.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:09:26.078 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:09:26.078 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:09:26.078 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:09:26.078 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:09:26.079 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:26.079 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:09:26.079 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:09:26.079 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:09:26.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:09:26.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:736: TEST_repair_stats_ec: local 'ceph_osd_args=--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T12:09:26.099 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:737: TEST_repair_stats_ec: expr 3 - 1 2026-02-17T12:09:26.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:737: TEST_repair_stats_ec: seq 0 2 2026-02-17T12:09:26.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:737: TEST_repair_stats_ec: for id in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:09:26.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats_ec: run_osd td/osd-scrub-repair 0 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:09:26.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:09:26.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:09:26.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:09:26.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:09:26.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:09:26.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:09:26.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:09:26.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:09:26.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:09:26.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:09:26.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:09:26.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:09:26.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:09:26.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:09:26.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:09:26.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:09:26.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:09:26.106 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:09:26.106 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:26.106 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:09:26.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:09:26.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:09:26.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:09:26.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:09:26.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:09:26.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:09:26.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:09:26.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:09:26.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:09:26.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:09:26.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:09:26.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:09:26.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T12:09:26.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:09:26.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:09:26.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=eda2ee03-6acd-4354-a1f3-dfa25e26345e 2026-02-17T12:09:26.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 eda2ee03-6acd-4354-a1f3-dfa25e26345e' 2026-02-17T12:09:26.109 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 eda2ee03-6acd-4354-a1f3-dfa25e26345e 2026-02-17T12:09:26.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:09:26.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB2WpRpZmG/BhAARkRbuK//GpyPZZCmJM3KYg== 2026-02-17T12:09:26.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB2WpRpZmG/BhAARkRbuK//GpyPZZCmJM3KYg=="}' 2026-02-17T12:09:26.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new eda2ee03-6acd-4354-a1f3-dfa25e26345e -i td/osd-scrub-repair/0/new.json 2026-02-17T12:09:26.207 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:09:26.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T12:09:26.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQB2WpRpZmG/BhAARkRbuK//GpyPZZCmJM3KYg== --osd-uuid eda2ee03-6acd-4354-a1f3-dfa25e26345e 2026-02-17T12:09:26.233 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:26.224+0000 7f8858d648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:26.234 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:26.228+0000 7f8858d648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:26.235 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:26.228+0000 7f8858d648c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:26.235 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:26.228+0000 7f8858d648c0 -1 bdev(0x565205e67c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:09:26.236 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:26.228+0000 7f8858d648c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T12:09:28.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T12:09:28.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:09:28.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:09:28.551 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:09:28.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:09:28.747 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:09:28.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:09:28.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:09:28.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:09:28.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:09:28.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:09:28.764 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:28.756+0000 7fae121228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:28.765 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:28.756+0000 7fae121228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:28.766 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:28.756+0000 7fae121228c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:28.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:09:28.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:09:28.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:09:28.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:09:28.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:09:28.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:28.903 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:09:28.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:09:28.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:28.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:09:29.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:30.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:30.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:30.059 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:09:30.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:09:30.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:30.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:09:30.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:30.284 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:30.275+0000 7fae121228c0 -1 Falling back to public interface 2026-02-17T12:09:31.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:31.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:31.213 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:09:31.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:09:31.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:31.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:09:31.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:09:32.368 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:09:32.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:32.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:09:32.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:33.322 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:33.311+0000 7fae121228c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:09:33.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:33.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:33.526 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:09:33.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:09:33.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:33.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:09:33.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:34.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:34.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:34.690 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:09:34.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:09:34.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:34.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:09:34.842 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2560274953,v1:127.0.0.1:6803/2560274953] [v2:127.0.0.1:6804/2560274953,v1:127.0.0.1:6805/2560274953] exists,up eda2ee03-6acd-4354-a1f3-dfa25e26345e 2026-02-17T12:09:34.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:09:34.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:09:34.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:09:34.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:737: TEST_repair_stats_ec: for id in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:09:34.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats_ec: run_osd td/osd-scrub-repair 1 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:09:34.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:09:34.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:09:34.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:09:34.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:09:34.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:09:34.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:09:34.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:09:34.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:09:34.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:09:34.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:09:34.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:09:34.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:09:34.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:09:34.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:09:34.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:09:34.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:09:34.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:09:34.847 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:09:34.847 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:34.848 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:09:34.848 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:09:34.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:09:34.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:09:34.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:09:34.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:09:34.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:09:34.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:09:34.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:09:34.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:09:34.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:09:34.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:09:34.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:09:34.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T12:09:34.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:09:34.849 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:09:34.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=4c1700cf-b522-4aab-8b68-9a016102e1d8 2026-02-17T12:09:34.850 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 4c1700cf-b522-4aab-8b68-9a016102e1d8 2026-02-17T12:09:34.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 4c1700cf-b522-4aab-8b68-9a016102e1d8' 2026-02-17T12:09:34.850 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:09:34.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB+WpRptSn0MhAAvM94TqDdTo8HtPbLk4kXdQ== 2026-02-17T12:09:34.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB+WpRptSn0MhAAvM94TqDdTo8HtPbLk4kXdQ=="}' 2026-02-17T12:09:34.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 4c1700cf-b522-4aab-8b68-9a016102e1d8 -i td/osd-scrub-repair/1/new.json 2026-02-17T12:09:35.016 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:09:35.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T12:09:35.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQB+WpRptSn0MhAAvM94TqDdTo8HtPbLk4kXdQ== --osd-uuid 4c1700cf-b522-4aab-8b68-9a016102e1d8 2026-02-17T12:09:35.043 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:35.035+0000 7f98884db8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:35.044 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:35.035+0000 7f98884db8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:35.045 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:35.035+0000 7f98884db8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:35.046 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:35.035+0000 7f98884db8c0 -1 bdev(0x56424ff5dc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:09:35.046 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:35.035+0000 7f98884db8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T12:09:37.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T12:09:37.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:09:37.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:09:37.345 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:09:37.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:09:37.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:09:37.536 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:09:37.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:09:37.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:09:37.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:09:37.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:09:37.558 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:37.551+0000 7efe0d5538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:37.560 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:37.551+0000 7efe0d5538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:37.561 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:37.551+0000 7efe0d5538c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:37.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:09:37.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:09:37.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:09:37.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:09:37.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:09:37.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:37.694 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:09:37.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:09:37.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:37.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:09:37.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:38.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:38.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:38.844 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:09:38.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:09:38.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:38.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:09:38.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:39.052 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:39.043+0000 7efe0d5538c0 -1 Falling back to public interface 2026-02-17T12:09:39.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:39.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:39.999 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:09:39.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:09:40.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:40.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:09:40.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:41.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:41.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:41.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:09:41.150 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:09:41.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:41.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:09:41.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:42.042 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:42.031+0000 7efe0d5538c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:09:42.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:42.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:42.300 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:09:42.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:09:42.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:42.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:09:42.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:43.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:43.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:43.455 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:09:43.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:09:43.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:43.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:09:43.603 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/508060101,v1:127.0.0.1:6811/508060101] [v2:127.0.0.1:6812/508060101,v1:127.0.0.1:6813/508060101] exists,up 4c1700cf-b522-4aab-8b68-9a016102e1d8 2026-02-17T12:09:43.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:09:43.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:09:43.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:09:43.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:737: TEST_repair_stats_ec: for id in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:09:43.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:738: TEST_repair_stats_ec: run_osd td/osd-scrub-repair 2 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:09:43.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:09:43.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:09:43.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:09:43.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:09:43.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T12:09:43.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:09:43.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:09:43.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:09:43.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:09:43.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T12:09:43.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T12:09:43.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:09:43.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:09:43.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:09:43.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:09:43.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:09:43.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:09:43.609 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:09:43.609 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:09:43.609 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:09:43.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:09:43.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:09:43.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:09:43.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:09:43.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:09:43.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:09:43.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:09:43.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:09:43.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:09:43.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:09:43.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:09:43.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:09:43.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T12:09:43.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T12:09:43.611 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:09:43.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a8bcd243-e363-4b8e-b543-5de4b96324e1 2026-02-17T12:09:43.611 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 a8bcd243-e363-4b8e-b543-5de4b96324e1 2026-02-17T12:09:43.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 a8bcd243-e363-4b8e-b543-5de4b96324e1' 2026-02-17T12:09:43.612 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:09:43.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCHWpRp89S4JBAAfx2FQuRD6n5GyKt6OXRcUQ== 2026-02-17T12:09:43.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCHWpRp89S4JBAAfx2FQuRD6n5GyKt6OXRcUQ=="}' 2026-02-17T12:09:43.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a8bcd243-e363-4b8e-b543-5de4b96324e1 -i td/osd-scrub-repair/2/new.json 2026-02-17T12:09:43.776 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:09:43.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T12:09:43.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 --mkfs --key AQCHWpRp89S4JBAAfx2FQuRD6n5GyKt6OXRcUQ== --osd-uuid a8bcd243-e363-4b8e-b543-5de4b96324e1 2026-02-17T12:09:43.802 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:43.795+0000 7f2600bd08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:43.804 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:43.795+0000 7f2600bd08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:43.805 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:43.795+0000 7f2600bd08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:43.805 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:43.795+0000 7f2600bd08c0 -1 bdev(0x56035c24bc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:09:43.805 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:43.795+0000 7f2600bd08c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T12:09:46.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T12:09:46.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:09:46.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:09:46.081 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T12:09:46.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:09:46.271 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T12:09:46.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:09:46.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:09:46.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:09:46.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:09:46.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:09:46.295 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:46.287+0000 7fdc65e218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:46.296 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:46.287+0000 7fdc65e218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:46.297 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:46.287+0000 7fdc65e218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:09:46.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:09:46.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:09:46.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:09:46.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:09:46.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:09:46.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:46.426 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:09:46.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:09:46.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:46.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:09:46.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:47.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:47.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:47.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:09:47.576 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:09:47.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:47.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:09:47.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:47.784 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:47.775+0000 7fdc65e218c0 -1 Falling back to public interface 2026-02-17T12:09:48.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:48.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:48.734 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:09:48.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:09:48.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:48.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:09:48.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:49.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:49.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:49.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:09:49.885 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:09:49.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:49.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:09:50.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:51.034 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:09:51.023+0000 7fdc65e218c0 -1 osd.2 0 log_to_monitors true 2026-02-17T12:09:51.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:51.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:51.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:09:51.037 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:09:51.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:51.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:09:51.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:09:52.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:09:52.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:09:52.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:09:52.189 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:09:52.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:09:52.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:09:52.338 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/967499374,v1:127.0.0.1:6819/967499374] [v2:127.0.0.1:6820/967499374,v1:127.0.0.1:6821/967499374] exists,up a8bcd243-e363-4b8e-b543-5de4b96324e1 2026-02-17T12:09:52.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:09:52.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:09:52.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:09:52.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:742: TEST_repair_stats_ec: create_ec_pool testpool false k=2 m=1 2026-02-17T12:09:52.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=testpool 2026-02-17T12:09:52.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T12:09:52.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=false 2026-02-17T12:09:52.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T12:09:52.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-17T12:09:52.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool testpool 1 1 erasure myprofile 2026-02-17T12:09:52.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create testpool 1 1 erasure myprofile 2026-02-17T12:09:52.815 INFO:tasks.workunit.client.0.trial128.stderr:pool 'testpool' created 2026-02-17T12:09:52.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:09:53.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' false = true ']' 2026-02-17T12:09:53.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T12:09:53.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:09:53.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:09:53.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:09:53.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:09:53.830 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:09:53.830 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:09:53.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:09:53.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:09:53.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:09:53.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:09:53.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:09:53.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:09:53.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:09:53.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:09:53.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:09:54.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:09:54.105 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:09:54.106 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:09:54.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:09:54.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:09:54.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:09:54.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T12:09:54.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T12:09:54.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T12:09:54.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:09:54.177 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:09:54.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T12:09:54.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T12:09:54.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-17T12:09:54.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:09:54.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:09:54.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542146 2026-02-17T12:09:54.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542146 2026-02-17T12:09:54.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-60129542146' 2026-02-17T12:09:54.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:09:54.319 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T12:09:54.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:09:54.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:09:54.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T12:09:54.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:09:54.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T12:09:54.325 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836485 2026-02-17T12:09:54.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T12:09:54.326 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:09:54.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-17T12:09:54.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:09:54.488 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T12:09:54.488 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:09:54.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:09:54.490 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T12:09:54.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:09:54.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T12:09:54.493 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672964 2026-02-17T12:09:54.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T12:09:54.493 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:09:54.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T12:09:54.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:09:54.653 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542146 2026-02-17T12:09:54.653 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:09:54.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:09:54.655 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542146 2026-02-17T12:09:54.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:09:54.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542146 2026-02-17T12:09:54.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-17T12:09:54.657 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542146 2026-02-17T12:09:54.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:09:54.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542146 -lt 60129542146 2026-02-17T12:09:54.819 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:09:54.819 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:09:54.819 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:09:55.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:09:55.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:09:55.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:09:55.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:09:55.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:09:55.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:09:55.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:09:55.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:09:55.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:09:55.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:09:55.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:09:55.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:09:55.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:09:55.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:09:55.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:09:55.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T12:09:55.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:745: TEST_repair_stats_ec: local payload=ABCDEF 2026-02-17T12:09:55.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:746: TEST_repair_stats_ec: echo ABCDEF 2026-02-17T12:09:55.338 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: seq 1 30 2026-02-17T12:09:55.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj1 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj2 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj3 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj4 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj5 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj6 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj7 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj8 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj9 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj10 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj11 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj12 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj13 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj14 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj15 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj16 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj17 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj18 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj19 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj20 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj21 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj22 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj23 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj24 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj25 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj26 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj27 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:55.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:55.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj28 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:56.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:56.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj29 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:56.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:747: TEST_repair_stats_ec: for i in $(seq 1 $OBJS) 2026-02-17T12:09:56.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:749: TEST_repair_stats_ec: rados --pool testpool put obj30 td/osd-scrub-repair/ORIGINAL 2026-02-17T12:09:56.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:754: TEST_repair_stats_ec: get_not_primary testpool obj1 2026-02-17T12:09:56.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=testpool 2026-02-17T12:09:56.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=obj1 2026-02-17T12:09:56.054 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary testpool obj1 2026-02-17T12:09:56.054 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T12:09:56.054 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T12:09:56.054 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool obj1 2026-02-17T12:09:56.054 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:09:56.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T12:09:56.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map testpool obj1 2026-02-17T12:09:56.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T12:09:56.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:754: TEST_repair_stats_ec: local other=0 2026-02-17T12:09:56.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:755: TEST_repair_stats_ec: get_pg testpool obj1 2026-02-17T12:09:56.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=testpool 2026-02-17T12:09:56.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=obj1 2026-02-17T12:09:56.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map testpool obj1 2026-02-17T12:09:56.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T12:09:56.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:755: TEST_repair_stats_ec: local pgid=1.0 2026-02-17T12:09:56.520 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:756: TEST_repair_stats_ec: get_primary testpool obj1 2026-02-17T12:09:56.521 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=testpool 2026-02-17T12:09:56.521 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T12:09:56.521 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map testpool obj1 2026-02-17T12:09:56.521 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:09:56.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:756: TEST_repair_stats_ec: local primary=1 2026-02-17T12:09:56.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:758: TEST_repair_stats_ec: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T12:09:56.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:09:56.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:09:56.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:09:56.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:09:56.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:09:57.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:09:57.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:759: TEST_repair_stats_ec: kill_daemons td/osd-scrub-repair TERM osd.1 2026-02-17T12:09:57.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:09:57.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:09:57.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:09:57.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:09:57.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:09:59.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:09:59.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: seq 1 26 2026-02-17T12:09:59.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:09:59.303 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 1 % 2 2026-02-17T12:09:59.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T12:09:59.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj1 remove 2026-02-17T12:09:59.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:09:59.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:09:59.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:09:59.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:09:59.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:09:59.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj1 remove 2026-02-17T12:09:59.945 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#1:ff7b1f36:::obj1:head# 2026-02-17T12:10:00.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:00.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 2 % 2 2026-02-17T12:10:00.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T12:10:00.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj2 remove 2026-02-17T12:10:00.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:00.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:00.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:10:00.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:00.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:10:00.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj2 remove 2026-02-17T12:10:01.140 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#1:104778fc:::obj2:head# 2026-02-17T12:10:01.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:01.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 3 % 2 2026-02-17T12:10:01.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T12:10:01.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj3 remove 2026-02-17T12:10:01.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:01.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:01.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:10:01.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:01.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:10:01.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj3 remove 2026-02-17T12:10:02.311 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#1:8dd16f86:::obj3:head# 2026-02-17T12:10:02.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:02.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 4 % 2 2026-02-17T12:10:02.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T12:10:02.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj4 remove 2026-02-17T12:10:02.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:02.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:02.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:10:02.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:02.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:10:02.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj4 remove 2026-02-17T12:10:03.515 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#1:0ee9ae15:::obj4:head# 2026-02-17T12:10:04.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:04.052 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 5 % 2 2026-02-17T12:10:04.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T12:10:04.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj5 remove 2026-02-17T12:10:04.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:04.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:04.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:10:04.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:04.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:10:04.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj5 remove 2026-02-17T12:10:04.691 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#1:c52c9666:::obj5:head# 2026-02-17T12:10:05.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:05.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 6 % 2 2026-02-17T12:10:05.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T12:10:05.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj6 remove 2026-02-17T12:10:05.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:05.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:05.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:10:05.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:05.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:10:05.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj6 remove 2026-02-17T12:10:05.867 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#1:233a42c1:::obj6:head# 2026-02-17T12:10:06.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:06.404 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 7 % 2 2026-02-17T12:10:06.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T12:10:06.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj7 remove 2026-02-17T12:10:06.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:06.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:06.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:10:06.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:06.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:10:06.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj7 remove 2026-02-17T12:10:07.040 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#1:9a09113a:::obj7:head# 2026-02-17T12:10:07.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:07.576 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 8 % 2 2026-02-17T12:10:07.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T12:10:07.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj8 remove 2026-02-17T12:10:07.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:07.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:07.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:10:07.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:07.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:10:07.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj8 remove 2026-02-17T12:10:08.212 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#1:a557efb1:::obj8:head# 2026-02-17T12:10:08.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:08.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 9 % 2 2026-02-17T12:10:08.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T12:10:08.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj9 remove 2026-02-17T12:10:08.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:08.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:08.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:10:08.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:08.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:10:08.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj9 remove 2026-02-17T12:10:09.407 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#1:fe351e27:::obj9:head# 2026-02-17T12:10:09.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:09.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 10 % 2 2026-02-17T12:10:09.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T12:10:09.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj10 remove 2026-02-17T12:10:09.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:09.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:09.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:10:09.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:09.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:10:09.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj10 remove 2026-02-17T12:10:10.571 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#1:d1337354:::obj10:head# 2026-02-17T12:10:11.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:11.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 11 % 2 2026-02-17T12:10:11.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T12:10:11.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj11 remove 2026-02-17T12:10:11.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:11.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:11.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:10:11.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:11.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:10:11.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj11 remove 2026-02-17T12:10:11.764 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#1:eeae2a94:::obj11:head# 2026-02-17T12:10:12.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:12.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 12 % 2 2026-02-17T12:10:12.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T12:10:12.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj12 remove 2026-02-17T12:10:12.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:12.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:12.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:10:12.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:12.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:10:12.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj12 remove 2026-02-17T12:10:12.951 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#1:af5f95cb:::obj12:head# 2026-02-17T12:10:13.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:13.492 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 13 % 2 2026-02-17T12:10:13.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T12:10:13.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj13 remove 2026-02-17T12:10:13.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:13.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:13.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:10:13.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:13.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:10:13.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj13 remove 2026-02-17T12:10:14.119 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#1:a61ad63e:::obj13:head# 2026-02-17T12:10:14.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:14.667 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 14 % 2 2026-02-17T12:10:14.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T12:10:14.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj14 remove 2026-02-17T12:10:14.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:14.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:14.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:10:14.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:14.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:10:14.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj14 remove 2026-02-17T12:10:15.323 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#1:e0a44829:::obj14:head# 2026-02-17T12:10:15.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:15.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 15 % 2 2026-02-17T12:10:15.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T12:10:15.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj15 remove 2026-02-17T12:10:15.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:15.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:15.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:10:15.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:15.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:10:15.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj15 remove 2026-02-17T12:10:16.487 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#1:ab946124:::obj15:head# 2026-02-17T12:10:17.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:17.025 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 16 % 2 2026-02-17T12:10:17.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T12:10:17.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj16 remove 2026-02-17T12:10:17.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:17.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:17.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:10:17.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:17.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:10:17.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj16 remove 2026-02-17T12:10:17.662 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#1:461f8b5e:::obj16:head# 2026-02-17T12:10:18.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:18.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 17 % 2 2026-02-17T12:10:18.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T12:10:18.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj17 remove 2026-02-17T12:10:18.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:18.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:18.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:10:18.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:18.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:10:18.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj17 remove 2026-02-17T12:10:18.835 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#1:24fc6e92:::obj17:head# 2026-02-17T12:10:19.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:19.373 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 18 % 2 2026-02-17T12:10:19.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T12:10:19.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj18 remove 2026-02-17T12:10:19.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:19.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:19.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:10:19.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:19.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:10:19.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj18 remove 2026-02-17T12:10:20.015 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#1:b9836a99:::obj18:head# 2026-02-17T12:10:20.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:20.551 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 19 % 2 2026-02-17T12:10:20.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T12:10:20.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj19 remove 2026-02-17T12:10:20.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:20.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:20.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:10:20.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:20.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:10:20.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj19 remove 2026-02-17T12:10:21.182 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#1:00b31b6c:::obj19:head# 2026-02-17T12:10:21.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:21.716 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 20 % 2 2026-02-17T12:10:21.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T12:10:21.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj20 remove 2026-02-17T12:10:21.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:21.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:21.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:10:21.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:21.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:10:21.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj20 remove 2026-02-17T12:10:22.363 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#1:24b5611b:::obj20:head# 2026-02-17T12:10:22.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:22.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 21 % 2 2026-02-17T12:10:22.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T12:10:22.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj21 remove 2026-02-17T12:10:22.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:22.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:22.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:10:22.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:22.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:10:22.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj21 remove 2026-02-17T12:10:23.535 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#1:1243fe1a:::obj21:head# 2026-02-17T12:10:24.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:24.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 22 % 2 2026-02-17T12:10:24.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T12:10:24.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj22 remove 2026-02-17T12:10:24.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:24.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:24.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:10:24.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:24.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:10:24.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj22 remove 2026-02-17T12:10:24.704 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#1:c0c49def:::obj22:head# 2026-02-17T12:10:25.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:25.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 23 % 2 2026-02-17T12:10:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T12:10:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj23 remove 2026-02-17T12:10:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:10:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:10:25.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj23 remove 2026-02-17T12:10:25.879 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#1:242c40e0:::obj23:head# 2026-02-17T12:10:26.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:26.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 24 % 2 2026-02-17T12:10:26.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T12:10:26.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj24 remove 2026-02-17T12:10:26.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:26.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:26.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:10:26.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:26.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:10:26.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj24 remove 2026-02-17T12:10:27.061 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#1:7eacede5:::obj24:head# 2026-02-17T12:10:27.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:27.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 25 % 2 2026-02-17T12:10:27.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=1 2026-02-17T12:10:27.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 1 obj25 remove 2026-02-17T12:10:27.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:27.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:27.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:10:27.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:27.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:10:27.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/1 obj25 remove 2026-02-17T12:10:28.231 INFO:tasks.workunit.client.0.trial128.stdout:remove 0#1:7109478f:::obj25:head# 2026-02-17T12:10:28.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:760: TEST_repair_stats_ec: for i in $(seq 1 $REPAIRS) 2026-02-17T12:10:28.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: expr 26 % 2 2026-02-17T12:10:28.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:763: TEST_repair_stats_ec: OSD=0 2026-02-17T12:10:28.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:764: TEST_repair_stats_ec: _objectstore_tool_nodown td/osd-scrub-repair 0 obj26 remove 2026-02-17T12:10:28.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:10:28.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:10:28.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:10:28.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:10:28.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:10:28.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 obj26 remove 2026-02-17T12:10:29.405 INFO:tasks.workunit.client.0.trial128.stdout:remove 1#1:f3c2593a:::obj26:head# 2026-02-17T12:10:29.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:766: TEST_repair_stats_ec: activate_osd td/osd-scrub-repair 1 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:10:29.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:10:29.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:10:29.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:10:29.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:10:29.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:10:29.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:10:29.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:10:29.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:10:29.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:10:29.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:10:29.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:10:29.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:10:29.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:10:29.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:10:29.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:10:29.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:10:29.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:10:29.945 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:10:29.945 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:29.945 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:10:29.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:10:29.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:10:29.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:10:29.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:10:29.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:10:29.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:10:29.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:10:29.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:10:29.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:10:29.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:10:29.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T12:10:29.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:10:29.946 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:10:29.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:10:29.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:10:29.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/1/whoami 2026-02-17T12:10:29.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:10:29.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:10:29.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:10:29.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:10:29.967 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:29.958+0000 7fd95b7438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:29.969 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:29.958+0000 7fd95b7438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:29.970 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:29.958+0000 7fd95b7438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:30.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:10:30.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:10:30.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:10:30.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:10:30.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:10:30.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:30.102 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:10:30.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:10:30.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:30.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:10:30.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:31.204 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:31.194+0000 7fd95b7438c0 -1 Falling back to public interface 2026-02-17T12:10:31.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:31.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:31.253 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:10:31.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:10:31.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:31.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:10:31.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:32.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:32.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:32.411 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:10:32.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:10:32.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:32.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:10:32.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:33.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:33.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:33.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:10:33.564 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:10:33.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:33.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:10:33.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:34.176 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:34.166+0000 7fd95b7438c0 -1 osd.1 22 log_to_monitors true 2026-02-17T12:10:34.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:34.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:34.714 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:10:34.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:10:34.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:34.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:10:34.862 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 26 up_thru 26 down_at 23 last_clean_interval [10,22) [v2:127.0.0.1:6802/3717733369,v1:127.0.0.1:6803/3717733369] [v2:127.0.0.1:6804/3717733369,v1:127.0.0.1:6805/3717733369] exists,up 4c1700cf-b522-4aab-8b68-9a016102e1d8 2026-02-17T12:10:34.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:10:34.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:10:34.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:10:34.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:767: TEST_repair_stats_ec: activate_osd td/osd-scrub-repair 0 --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:10:34.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:10:34.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:10:34.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:10:34.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:10:34.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:10:34.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:10:34.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:10:34.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:10:34.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:10:34.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:10:34.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:10:34.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:10:34.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:10:34.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:10:34.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:10:34.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:10:34.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:10:34.867 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:10:34.867 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:34.867 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:10:34.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:10:34.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:10:34.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:10:34.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:10:34.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:10:34.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:10:34.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:10:34.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:10:34.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:10:34.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:10:34.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0' 2026-02-17T12:10:34.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:10:34.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:10:34.869 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:10:34.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd-scrub-interval-randomize-ratio=0 2026-02-17T12:10:34.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T12:10:34.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:10:34.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:10:34.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:10:34.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:10:34.884 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:34.874+0000 7f49aefa68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:34.886 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:34.874+0000 7f49aefa68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:34.887 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:34.878+0000 7f49aefa68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:35.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:10:35.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:10:35.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:10:35.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:10:35.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:10:35.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:35.023 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:10:35.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:10:35.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:35.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:10:35.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:36.124 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:36.114+0000 7f49aefa68c0 -1 Falling back to public interface 2026-02-17T12:10:36.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:36.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:36.173 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:10:36.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:10:36.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:36.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:10:36.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:37.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:37.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:37.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:10:37.322 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:10:37.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:37.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:10:37.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:38.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:38.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:38.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:10:38.472 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:10:38.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:38.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:10:38.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:39.099 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:39.090+0000 7f49aefa68c0 -1 osd.0 20 log_to_monitors true 2026-02-17T12:10:39.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:39.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:39.622 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:10:39.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:10:39.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:39.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:10:39.773 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 30 up_thru 0 down_at 21 last_clean_interval [5,20) [v2:127.0.0.1:6810/3148958449,v1:127.0.0.1:6811/3148958449] [v2:127.0.0.1:6812/3148958449,v1:127.0.0.1:6813/3148958449] exists,up eda2ee03-6acd-4354-a1f3-dfa25e26345e 2026-02-17T12:10:39.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:10:39.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:10:39.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:10:39.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:768: TEST_repair_stats_ec: wait_for_clean 2026-02-17T12:10:39.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:10:39.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:10:39.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:10:39.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:10:39.776 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:10:39.776 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:10:39.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:10:39.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:10:39.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:10:39.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:10:39.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:10:39.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:10:39.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:10:39.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:10:39.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:10:40.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:10:40.056 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:10:40.056 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:10:40.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:10:40.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:10:40.057 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:10:40.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018882 2026-02-17T12:10:40.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018882 2026-02-17T12:10:40.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018882' 2026-02-17T12:10:40.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:10:40.128 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:10:40.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149699 2026-02-17T12:10:40.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149699 2026-02-17T12:10:40.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018882 1-111669149699' 2026-02-17T12:10:40.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:10:40.201 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:10:40.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542156 2026-02-17T12:10:40.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542156 2026-02-17T12:10:40.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018882 1-111669149699 2-60129542156' 2026-02-17T12:10:40.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:10:40.272 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-128849018882 2026-02-17T12:10:40.272 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:10:40.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:10:40.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-128849018882 2026-02-17T12:10:40.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:10:40.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018882 2026-02-17T12:10:40.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 128849018882' 2026-02-17T12:10:40.277 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 128849018882 2026-02-17T12:10:40.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:10:40.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018882 -lt 128849018882 2026-02-17T12:10:40.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:10:40.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-111669149699 2026-02-17T12:10:40.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:10:40.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:10:40.444 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-111669149699 2026-02-17T12:10:40.445 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:10:40.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149699 2026-02-17T12:10:40.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 111669149699' 2026-02-17T12:10:40.447 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 111669149699 2026-02-17T12:10:40.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:10:40.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-17T12:10:40.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:10:40.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542156 2026-02-17T12:10:40.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:10:40.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:10:40.604 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542156 2026-02-17T12:10:40.605 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:10:40.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542156 2026-02-17T12:10:40.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542156' 2026-02-17T12:10:40.607 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542156 2026-02-17T12:10:40.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:10:40.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542156 -lt 60129542156 2026-02-17T12:10:40.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:10:40.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:10:40.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:10:40.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:10:40.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:10:40.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:10:40.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:10:40.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:10:40.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:10:40.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:10:40.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:10:41.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:10:41.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:10:41.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:10:41.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:10:41.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:10:41.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:10:41.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:10:41.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:770: TEST_repair_stats_ec: repair 1.0 2026-02-17T12:10:41.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.0 2026-02-17T12:10:41.276 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.0 2026-02-17T12:10:41.276 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:10:41.276 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:10:41.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:10:41.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:10:41.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T12:09:52.807925+0000 2026-02-17T12:10:41.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.0 2026-02-17T12:10:41.570 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 1.0s0 on osd.1 to repair 2026-02-17T12:10:41.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.0 2026-02-17T12:09:52.807925+0000 2026-02-17T12:10:41.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T12:10:41.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:09:52.807925+0000 2026-02-17T12:10:41.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:10:41.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:10:41.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:10:41.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:10:41.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:10:41.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:10:41.582 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:10:41.582 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:10:41.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:09:52.807925+0000 '>' 2026-02-17T12:09:52.807925+0000 2026-02-17T12:10:41.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:10:42.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:10:42.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:10:42.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:10:42.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:10:42.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:10:42.732 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:10:42.732 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:10:42.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:09:52.807925+0000 '>' 2026-02-17T12:09:52.807925+0000 2026-02-17T12:10:42.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:10:43.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:10:43.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:10:43.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:10:43.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:10:43.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:10:43.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:10:43.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:10:44.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:09:52.807925+0000 '>' 2026-02-17T12:09:52.807925+0000 2026-02-17T12:10:44.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:10:45.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:10:45.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:10:45.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:10:45.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:10:45.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:10:45.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:10:45.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:10:45.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:10:42.183355+0000 '>' 2026-02-17T12:09:52.807925+0000 2026-02-17T12:10:45.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:10:45.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:771: TEST_repair_stats_ec: wait_for_clean 2026-02-17T12:10:45.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:10:45.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:10:45.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:10:45.183 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:10:45.184 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:10:45.184 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:10:45.184 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:10:45.184 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:10:45.184 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:10:45.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:10:45.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:10:45.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:10:45.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:10:45.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:10:45.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:10:45.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:10:45.458 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:10:45.458 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:10:45.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:10:45.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:10:45.458 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:10:45.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018884 2026-02-17T12:10:45.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018884 2026-02-17T12:10:45.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018884' 2026-02-17T12:10:45.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:10:45.533 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:10:45.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149701 2026-02-17T12:10:45.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149701 2026-02-17T12:10:45.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018884 1-111669149701' 2026-02-17T12:10:45.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:10:45.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:10:45.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542158 2026-02-17T12:10:45.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542158 2026-02-17T12:10:45.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018884 1-111669149701 2-60129542158' 2026-02-17T12:10:45.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:10:45.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-128849018884 2026-02-17T12:10:45.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:10:45.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:10:45.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-128849018884 2026-02-17T12:10:45.683 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:10:45.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018884 2026-02-17T12:10:45.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 128849018884' 2026-02-17T12:10:45.684 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 128849018884 2026-02-17T12:10:45.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:10:45.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018883 -lt 128849018884 2026-02-17T12:10:45.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:10:46.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:10:46.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:10:47.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018884 -lt 128849018884 2026-02-17T12:10:47.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:10:47.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-111669149701 2026-02-17T12:10:47.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:10:47.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:10:47.007 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-111669149701 2026-02-17T12:10:47.008 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:10:47.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149701 2026-02-17T12:10:47.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 111669149701' 2026-02-17T12:10:47.010 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 111669149701 2026-02-17T12:10:47.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:10:47.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149701 -lt 111669149701 2026-02-17T12:10:47.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:10:47.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542158 2026-02-17T12:10:47.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:10:47.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:10:47.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542158 2026-02-17T12:10:47.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:10:47.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542158 2026-02-17T12:10:47.181 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542158 2026-02-17T12:10:47.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542158' 2026-02-17T12:10:47.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:10:47.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542158 -lt 60129542158 2026-02-17T12:10:47.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:10:47.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:10:47.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:10:47.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:10:47.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:10:47.527 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:10:47.527 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:10:47.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:10:47.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:10:47.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:10:47.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:10:47.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:10:47.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:10:47.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:10:47.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:10:47.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:10:47.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:10:47.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:10:47.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:772: TEST_repair_stats_ec: ceph pg dump pgs 2026-02-17T12:10:48.012 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T12:10:48.012 INFO:tasks.workunit.client.0.trial128.stdout:1.0 30 0 0 0 0 210 0 0 30 0 30 active+clean 2026-02-17T12:10:42.220533+0000 20'30 31:137 [1,0,2] 1 [1,0,2] 1 20'30 2026-02-17T12:10:42.183355+0000 20'30 2026-02-17T12:10:42.183355+0000 0 1 periodic scrub scheduled @ 2026-02-18T12:10:42.183355+0000 17 0 2026-02-17T12:10:48.013 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T12:10:48.013 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T12:10:48.013 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:10:48.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:773: TEST_repair_stats_ec: flush_pg_stats 2026-02-17T12:10:48.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:10:48.021 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:10:48.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:10:48.186 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:10:48.187 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:10:48.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:10:48.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:10:48.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:10:48.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018885 2026-02-17T12:10:48.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018885 2026-02-17T12:10:48.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018885' 2026-02-17T12:10:48.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:10:48.261 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:10:48.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149702 2026-02-17T12:10:48.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149702 2026-02-17T12:10:48.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018885 1-111669149702' 2026-02-17T12:10:48.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:10:48.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:10:48.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542160 2026-02-17T12:10:48.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542160 2026-02-17T12:10:48.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018885 1-111669149702 2-60129542160' 2026-02-17T12:10:48.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:10:48.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-128849018885 2026-02-17T12:10:48.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:10:48.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:10:48.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-128849018885 2026-02-17T12:10:48.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:10:48.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018885 2026-02-17T12:10:48.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 128849018885' 2026-02-17T12:10:48.410 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 128849018885 2026-02-17T12:10:48.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:10:48.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018885 -lt 128849018885 2026-02-17T12:10:48.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:10:48.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-111669149702 2026-02-17T12:10:48.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:10:48.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:10:48.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-111669149702 2026-02-17T12:10:48.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:10:48.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149702 2026-02-17T12:10:48.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 111669149702' 2026-02-17T12:10:48.573 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 111669149702 2026-02-17T12:10:48.573 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:10:48.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-17T12:10:48.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:10:48.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542160 2026-02-17T12:10:48.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:10:48.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:10:48.740 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542160 2026-02-17T12:10:48.740 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:10:48.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542160 2026-02-17T12:10:48.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542160' 2026-02-17T12:10:48.742 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542160 2026-02-17T12:10:48.742 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:10:48.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542159 -lt 60129542160 2026-02-17T12:10:48.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:10:49.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:10:49.898 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:10:50.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542159 -lt 60129542160 2026-02-17T12:10:50.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:10:51.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:10:51.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:10:51.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542160 -lt 60129542160 2026-02-17T12:10:51.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:776: TEST_repair_stats_ec: ceph pg 1.0 query 2026-02-17T12:10:51.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:776: TEST_repair_stats_ec: jq .info.stats.stat_sum 2026-02-17T12:10:51.280 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:10:51.281 INFO:tasks.workunit.client.0.trial128.stdout: "num_bytes": 210, 2026-02-17T12:10:51.281 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects": 30, 2026-02-17T12:10:51.281 INFO:tasks.workunit.client.0.trial128.stdout: "num_object_clones": 0, 2026-02-17T12:10:51.281 INFO:tasks.workunit.client.0.trial128.stdout: "num_object_copies": 90, 2026-02-17T12:10:51.281 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_missing_on_primary": 0, 2026-02-17T12:10:51.281 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_missing": 0, 2026-02-17T12:10:51.281 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_degraded": 0, 2026-02-17T12:10:51.281 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_misplaced": 0, 2026-02-17T12:10:51.282 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_unfound": 0, 2026-02-17T12:10:51.282 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_dirty": 30, 2026-02-17T12:10:51.282 INFO:tasks.workunit.client.0.trial128.stdout: "num_whiteouts": 0, 2026-02-17T12:10:51.282 INFO:tasks.workunit.client.0.trial128.stdout: "num_read": 0, 2026-02-17T12:10:51.282 INFO:tasks.workunit.client.0.trial128.stdout: "num_read_kb": 0, 2026-02-17T12:10:51.282 INFO:tasks.workunit.client.0.trial128.stdout: "num_write": 30, 2026-02-17T12:10:51.282 INFO:tasks.workunit.client.0.trial128.stdout: "num_write_kb": 30, 2026-02-17T12:10:51.282 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrub_errors": 0, 2026-02-17T12:10:51.282 INFO:tasks.workunit.client.0.trial128.stdout: "num_shallow_scrub_errors": 0, 2026-02-17T12:10:51.282 INFO:tasks.workunit.client.0.trial128.stdout: "num_deep_scrub_errors": 0, 2026-02-17T12:10:51.282 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_recovered": 26, 2026-02-17T12:10:51.282 INFO:tasks.workunit.client.0.trial128.stdout: "num_bytes_recovered": 182, 2026-02-17T12:10:51.282 INFO:tasks.workunit.client.0.trial128.stdout: "num_keys_recovered": 0, 2026-02-17T12:10:51.283 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_omap": 0, 2026-02-17T12:10:51.283 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_hit_set_archive": 0, 2026-02-17T12:10:51.283 INFO:tasks.workunit.client.0.trial128.stdout: "num_bytes_hit_set_archive": 0, 2026-02-17T12:10:51.283 INFO:tasks.workunit.client.0.trial128.stdout: "num_flush": 0, 2026-02-17T12:10:51.283 INFO:tasks.workunit.client.0.trial128.stdout: "num_flush_kb": 0, 2026-02-17T12:10:51.283 INFO:tasks.workunit.client.0.trial128.stdout: "num_evict": 0, 2026-02-17T12:10:51.283 INFO:tasks.workunit.client.0.trial128.stdout: "num_evict_kb": 0, 2026-02-17T12:10:51.283 INFO:tasks.workunit.client.0.trial128.stdout: "num_promote": 0, 2026-02-17T12:10:51.283 INFO:tasks.workunit.client.0.trial128.stdout: "num_flush_mode_high": 0, 2026-02-17T12:10:51.283 INFO:tasks.workunit.client.0.trial128.stdout: "num_flush_mode_low": 0, 2026-02-17T12:10:51.283 INFO:tasks.workunit.client.0.trial128.stdout: "num_evict_mode_some": 0, 2026-02-17T12:10:51.283 INFO:tasks.workunit.client.0.trial128.stdout: "num_evict_mode_full": 0, 2026-02-17T12:10:51.284 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_pinned": 0, 2026-02-17T12:10:51.284 INFO:tasks.workunit.client.0.trial128.stdout: "num_legacy_snapsets": 0, 2026-02-17T12:10:51.284 INFO:tasks.workunit.client.0.trial128.stdout: "num_large_omap_objects": 0, 2026-02-17T12:10:51.284 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_manifest": 0, 2026-02-17T12:10:51.284 INFO:tasks.workunit.client.0.trial128.stdout: "num_omap_bytes": 0, 2026-02-17T12:10:51.284 INFO:tasks.workunit.client.0.trial128.stdout: "num_omap_keys": 0, 2026-02-17T12:10:51.284 INFO:tasks.workunit.client.0.trial128.stdout: "num_objects_repaired": 26 2026-02-17T12:10:51.284 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:10:51.285 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:777: TEST_repair_stats_ec: ceph pg 1.0 query 2026-02-17T12:10:51.285 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:777: TEST_repair_stats_ec: jq .info.stats.stat_sum.num_objects_repaired 2026-02-17T12:10:51.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:777: TEST_repair_stats_ec: COUNT=26 2026-02-17T12:10:51.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:778: TEST_repair_stats_ec: test 26 = 26 2026-02-17T12:10:51.356 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:780: TEST_repair_stats_ec: expr 3 - 1 2026-02-17T12:10:51.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:780: TEST_repair_stats_ec: seq 0 2 2026-02-17T12:10:51.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:780: TEST_repair_stats_ec: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:10:51.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:781: TEST_repair_stats_ec: '[' 0 = 0 -o 0 = 1 ']' 2026-02-17T12:10:51.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:782: TEST_repair_stats_ec: expr 26 / 2 2026-02-17T12:10:51.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:782: TEST_repair_stats_ec: repair=13 2026-02-17T12:10:51.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:787: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-17T12:10:51.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:787: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 0 )' 2026-02-17T12:10:51.504 INFO:tasks.workunit.client.0.trial128.stderr:dumped all 2026-02-17T12:10:51.512 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:10:51.512 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:10:51.512 INFO:tasks.workunit.client.0.trial128.stdout: "up_from": 30, 2026-02-17T12:10:51.513 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 128849018886, 2026-02-17T12:10:51.513 INFO:tasks.workunit.client.0.trial128.stdout: "num_pgs": 1, 2026-02-17T12:10:51.513 INFO:tasks.workunit.client.0.trial128.stdout: "num_osds": 1, 2026-02-17T12:10:51.513 INFO:tasks.workunit.client.0.trial128.stdout: "num_per_pool_osds": 1, 2026-02-17T12:10:51.513 INFO:tasks.workunit.client.0.trial128.stdout: "num_per_pool_omap_osds": 1, 2026-02-17T12:10:51.513 INFO:tasks.workunit.client.0.trial128.stdout: "kb": 104857600, 2026-02-17T12:10:51.513 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used": 28488, 2026-02-17T12:10:51.513 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_data": 376, 2026-02-17T12:10:51.513 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_omap": 4, 2026-02-17T12:10:51.513 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_meta": 28091, 2026-02-17T12:10:51.513 INFO:tasks.workunit.client.0.trial128.stdout: "kb_avail": 104829112, 2026-02-17T12:10:51.513 INFO:tasks.workunit.client.0.trial128.stdout: "statfs": { 2026-02-17T12:10:51.514 INFO:tasks.workunit.client.0.trial128.stdout: "total": 107374182400, 2026-02-17T12:10:51.514 INFO:tasks.workunit.client.0.trial128.stdout: "available": 107345010688, 2026-02-17T12:10:51.514 INFO:tasks.workunit.client.0.trial128.stdout: "internally_reserved": 0, 2026-02-17T12:10:51.514 INFO:tasks.workunit.client.0.trial128.stdout: "allocated": 385024, 2026-02-17T12:10:51.514 INFO:tasks.workunit.client.0.trial128.stdout: "data_stored": 207175, 2026-02-17T12:10:51.514 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed": 0, 2026-02-17T12:10:51.514 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed_allocated": 0, 2026-02-17T12:10:51.514 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed_original": 0, 2026-02-17T12:10:51.514 INFO:tasks.workunit.client.0.trial128.stdout: "omap_allocated": 4399, 2026-02-17T12:10:51.514 INFO:tasks.workunit.client.0.trial128.stdout: "internal_metadata": 28765905 2026-02-17T12:10:51.514 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:10:51.514 INFO:tasks.workunit.client.0.trial128.stdout: "hb_peers": [ 2026-02-17T12:10:51.515 INFO:tasks.workunit.client.0.trial128.stdout: 1, 2026-02-17T12:10:51.515 INFO:tasks.workunit.client.0.trial128.stdout: 2 2026-02-17T12:10:51.515 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:10:51.515 INFO:tasks.workunit.client.0.trial128.stdout: "snap_trim_queue_len": 0, 2026-02-17T12:10:51.515 INFO:tasks.workunit.client.0.trial128.stdout: "num_snap_trimming": 0, 2026-02-17T12:10:51.515 INFO:tasks.workunit.client.0.trial128.stdout: "num_shards_repaired": 13, 2026-02-17T12:10:51.515 INFO:tasks.workunit.client.0.trial128.stdout: "op_queue_age_hist": { 2026-02-17T12:10:51.515 INFO:tasks.workunit.client.0.trial128.stdout: "histogram": [], 2026-02-17T12:10:51.515 INFO:tasks.workunit.client.0.trial128.stdout: "upper_bound": 1 2026-02-17T12:10:51.515 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:10:51.515 INFO:tasks.workunit.client.0.trial128.stdout: "perf_stat": { 2026-02-17T12:10:51.515 INFO:tasks.workunit.client.0.trial128.stdout: "commit_latency_ms": 0, 2026-02-17T12:10:51.515 INFO:tasks.workunit.client.0.trial128.stdout: "apply_latency_ms": 0, 2026-02-17T12:10:51.516 INFO:tasks.workunit.client.0.trial128.stdout: "commit_latency_ns": 0, 2026-02-17T12:10:51.516 INFO:tasks.workunit.client.0.trial128.stdout: "apply_latency_ns": 0 2026-02-17T12:10:51.516 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:10:51.516 INFO:tasks.workunit.client.0.trial128.stdout: "alerts": [] 2026-02-17T12:10:51.516 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:10:51.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-17T12:10:51.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 0 ).num_shards_repaired' 2026-02-17T12:10:51.651 INFO:tasks.workunit.client.0.trial128.stderr:dumped all 2026-02-17T12:10:51.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: COUNT=13 2026-02-17T12:10:51.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:789: TEST_repair_stats_ec: test 13 = 13 2026-02-17T12:10:51.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:780: TEST_repair_stats_ec: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:10:51.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:781: TEST_repair_stats_ec: '[' 1 = 0 -o 1 = 1 ']' 2026-02-17T12:10:51.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:782: TEST_repair_stats_ec: expr 26 / 2 2026-02-17T12:10:51.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:782: TEST_repair_stats_ec: repair=13 2026-02-17T12:10:51.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:787: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-17T12:10:51.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:787: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 1 )' 2026-02-17T12:10:51.796 INFO:tasks.workunit.client.0.trial128.stderr:dumped all 2026-02-17T12:10:51.804 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:10:51.805 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:10:51.805 INFO:tasks.workunit.client.0.trial128.stdout: "up_from": 26, 2026-02-17T12:10:51.805 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 111669149703, 2026-02-17T12:10:51.805 INFO:tasks.workunit.client.0.trial128.stdout: "num_pgs": 1, 2026-02-17T12:10:51.805 INFO:tasks.workunit.client.0.trial128.stdout: "num_osds": 1, 2026-02-17T12:10:51.805 INFO:tasks.workunit.client.0.trial128.stdout: "num_per_pool_osds": 1, 2026-02-17T12:10:51.805 INFO:tasks.workunit.client.0.trial128.stdout: "num_per_pool_omap_osds": 1, 2026-02-17T12:10:51.805 INFO:tasks.workunit.client.0.trial128.stdout: "kb": 104857600, 2026-02-17T12:10:51.805 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used": 28488, 2026-02-17T12:10:51.805 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_data": 376, 2026-02-17T12:10:51.806 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_omap": 4, 2026-02-17T12:10:51.806 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_meta": 28091, 2026-02-17T12:10:51.806 INFO:tasks.workunit.client.0.trial128.stdout: "kb_avail": 104829112, 2026-02-17T12:10:51.806 INFO:tasks.workunit.client.0.trial128.stdout: "statfs": { 2026-02-17T12:10:51.806 INFO:tasks.workunit.client.0.trial128.stdout: "total": 107374182400, 2026-02-17T12:10:51.806 INFO:tasks.workunit.client.0.trial128.stdout: "available": 107345010688, 2026-02-17T12:10:51.806 INFO:tasks.workunit.client.0.trial128.stdout: "internally_reserved": 0, 2026-02-17T12:10:51.806 INFO:tasks.workunit.client.0.trial128.stdout: "allocated": 385024, 2026-02-17T12:10:51.806 INFO:tasks.workunit.client.0.trial128.stdout: "data_stored": 207175, 2026-02-17T12:10:51.806 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed": 0, 2026-02-17T12:10:51.806 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed_allocated": 0, 2026-02-17T12:10:51.806 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed_original": 0, 2026-02-17T12:10:51.807 INFO:tasks.workunit.client.0.trial128.stdout: "omap_allocated": 4395, 2026-02-17T12:10:51.807 INFO:tasks.workunit.client.0.trial128.stdout: "internal_metadata": 28765909 2026-02-17T12:10:51.807 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:10:51.807 INFO:tasks.workunit.client.0.trial128.stdout: "hb_peers": [ 2026-02-17T12:10:51.807 INFO:tasks.workunit.client.0.trial128.stdout: 0, 2026-02-17T12:10:51.807 INFO:tasks.workunit.client.0.trial128.stdout: 2 2026-02-17T12:10:51.807 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:10:51.807 INFO:tasks.workunit.client.0.trial128.stdout: "snap_trim_queue_len": 0, 2026-02-17T12:10:51.807 INFO:tasks.workunit.client.0.trial128.stdout: "num_snap_trimming": 0, 2026-02-17T12:10:51.807 INFO:tasks.workunit.client.0.trial128.stdout: "num_shards_repaired": 13, 2026-02-17T12:10:51.807 INFO:tasks.workunit.client.0.trial128.stdout: "op_queue_age_hist": { 2026-02-17T12:10:51.807 INFO:tasks.workunit.client.0.trial128.stdout: "histogram": [], 2026-02-17T12:10:51.807 INFO:tasks.workunit.client.0.trial128.stdout: "upper_bound": 1 2026-02-17T12:10:51.808 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:10:51.808 INFO:tasks.workunit.client.0.trial128.stdout: "perf_stat": { 2026-02-17T12:10:51.808 INFO:tasks.workunit.client.0.trial128.stdout: "commit_latency_ms": 0, 2026-02-17T12:10:51.808 INFO:tasks.workunit.client.0.trial128.stdout: "apply_latency_ms": 0, 2026-02-17T12:10:51.808 INFO:tasks.workunit.client.0.trial128.stdout: "commit_latency_ns": 0, 2026-02-17T12:10:51.808 INFO:tasks.workunit.client.0.trial128.stdout: "apply_latency_ns": 0 2026-02-17T12:10:51.808 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:10:51.808 INFO:tasks.workunit.client.0.trial128.stdout: "alerts": [] 2026-02-17T12:10:51.808 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:10:51.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-17T12:10:51.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 1 ).num_shards_repaired' 2026-02-17T12:10:51.953 INFO:tasks.workunit.client.0.trial128.stderr:dumped all 2026-02-17T12:10:51.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: COUNT=13 2026-02-17T12:10:51.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:789: TEST_repair_stats_ec: test 13 = 13 2026-02-17T12:10:51.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:780: TEST_repair_stats_ec: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:10:51.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:781: TEST_repair_stats_ec: '[' 2 = 0 -o 2 = 1 ']' 2026-02-17T12:10:51.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:784: TEST_repair_stats_ec: repair=0 2026-02-17T12:10:51.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:787: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-17T12:10:51.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:787: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 2 )' 2026-02-17T12:10:52.101 INFO:tasks.workunit.client.0.trial128.stderr:dumped all 2026-02-17T12:10:52.110 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:10:52.110 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 2, 2026-02-17T12:10:52.111 INFO:tasks.workunit.client.0.trial128.stdout: "up_from": 14, 2026-02-17T12:10:52.111 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 60129542160, 2026-02-17T12:10:52.111 INFO:tasks.workunit.client.0.trial128.stdout: "num_pgs": 1, 2026-02-17T12:10:52.111 INFO:tasks.workunit.client.0.trial128.stdout: "num_osds": 1, 2026-02-17T12:10:52.111 INFO:tasks.workunit.client.0.trial128.stdout: "num_per_pool_osds": 1, 2026-02-17T12:10:52.111 INFO:tasks.workunit.client.0.trial128.stdout: "num_per_pool_omap_osds": 1, 2026-02-17T12:10:52.111 INFO:tasks.workunit.client.0.trial128.stdout: "kb": 104857600, 2026-02-17T12:10:52.111 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used": 27348, 2026-02-17T12:10:52.111 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_data": 376, 2026-02-17T12:10:52.111 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_omap": 1, 2026-02-17T12:10:52.111 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_meta": 26942, 2026-02-17T12:10:52.112 INFO:tasks.workunit.client.0.trial128.stdout: "kb_avail": 104830252, 2026-02-17T12:10:52.112 INFO:tasks.workunit.client.0.trial128.stdout: "statfs": { 2026-02-17T12:10:52.112 INFO:tasks.workunit.client.0.trial128.stdout: "total": 107374182400, 2026-02-17T12:10:52.112 INFO:tasks.workunit.client.0.trial128.stdout: "available": 107346178048, 2026-02-17T12:10:52.112 INFO:tasks.workunit.client.0.trial128.stdout: "internally_reserved": 0, 2026-02-17T12:10:52.112 INFO:tasks.workunit.client.0.trial128.stdout: "allocated": 385024, 2026-02-17T12:10:52.112 INFO:tasks.workunit.client.0.trial128.stdout: "data_stored": 207175, 2026-02-17T12:10:52.112 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed": 0, 2026-02-17T12:10:52.112 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed_allocated": 0, 2026-02-17T12:10:52.112 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed_original": 0, 2026-02-17T12:10:52.112 INFO:tasks.workunit.client.0.trial128.stdout: "omap_allocated": 1594, 2026-02-17T12:10:52.112 INFO:tasks.workunit.client.0.trial128.stdout: "internal_metadata": 27589062 2026-02-17T12:10:52.113 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:10:52.113 INFO:tasks.workunit.client.0.trial128.stdout: "hb_peers": [ 2026-02-17T12:10:52.113 INFO:tasks.workunit.client.0.trial128.stdout: 0, 2026-02-17T12:10:52.113 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:10:52.113 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:10:52.113 INFO:tasks.workunit.client.0.trial128.stdout: "snap_trim_queue_len": 0, 2026-02-17T12:10:52.113 INFO:tasks.workunit.client.0.trial128.stdout: "num_snap_trimming": 0, 2026-02-17T12:10:52.113 INFO:tasks.workunit.client.0.trial128.stdout: "num_shards_repaired": 0, 2026-02-17T12:10:52.113 INFO:tasks.workunit.client.0.trial128.stdout: "op_queue_age_hist": { 2026-02-17T12:10:52.113 INFO:tasks.workunit.client.0.trial128.stdout: "histogram": [], 2026-02-17T12:10:52.113 INFO:tasks.workunit.client.0.trial128.stdout: "upper_bound": 1 2026-02-17T12:10:52.113 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:10:52.113 INFO:tasks.workunit.client.0.trial128.stdout: "perf_stat": { 2026-02-17T12:10:52.114 INFO:tasks.workunit.client.0.trial128.stdout: "commit_latency_ms": 0, 2026-02-17T12:10:52.114 INFO:tasks.workunit.client.0.trial128.stdout: "apply_latency_ms": 0, 2026-02-17T12:10:52.114 INFO:tasks.workunit.client.0.trial128.stdout: "commit_latency_ns": 0, 2026-02-17T12:10:52.114 INFO:tasks.workunit.client.0.trial128.stdout: "apply_latency_ns": 0 2026-02-17T12:10:52.114 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:10:52.114 INFO:tasks.workunit.client.0.trial128.stdout: "alerts": [] 2026-02-17T12:10:52.114 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:10:52.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-17T12:10:52.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: jq '.pg_map.osd_stats[] | select(.osd == 2 ).num_shards_repaired' 2026-02-17T12:10:52.258 INFO:tasks.workunit.client.0.trial128.stderr:dumped all 2026-02-17T12:10:52.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:788: TEST_repair_stats_ec: COUNT=0 2026-02-17T12:10:52.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:789: TEST_repair_stats_ec: test 0 = 0 2026-02-17T12:10:52.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:792: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-17T12:10:52.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:792: TEST_repair_stats_ec: jq .pg_map.osd_stats_sum 2026-02-17T12:10:52.408 INFO:tasks.workunit.client.0.trial128.stderr:dumped all 2026-02-17T12:10:52.416 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:10:52.416 INFO:tasks.workunit.client.0.trial128.stdout: "up_from": 0, 2026-02-17T12:10:52.416 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 0, 2026-02-17T12:10:52.416 INFO:tasks.workunit.client.0.trial128.stdout: "num_pgs": 3, 2026-02-17T12:10:52.417 INFO:tasks.workunit.client.0.trial128.stdout: "num_osds": 3, 2026-02-17T12:10:52.417 INFO:tasks.workunit.client.0.trial128.stdout: "num_per_pool_osds": 3, 2026-02-17T12:10:52.417 INFO:tasks.workunit.client.0.trial128.stdout: "num_per_pool_omap_osds": 3, 2026-02-17T12:10:52.417 INFO:tasks.workunit.client.0.trial128.stdout: "kb": 314572800, 2026-02-17T12:10:52.417 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used": 84324, 2026-02-17T12:10:52.417 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_data": 1128, 2026-02-17T12:10:52.417 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_omap": 10, 2026-02-17T12:10:52.417 INFO:tasks.workunit.client.0.trial128.stdout: "kb_used_meta": 83125, 2026-02-17T12:10:52.417 INFO:tasks.workunit.client.0.trial128.stdout: "kb_avail": 314488476, 2026-02-17T12:10:52.417 INFO:tasks.workunit.client.0.trial128.stdout: "statfs": { 2026-02-17T12:10:52.417 INFO:tasks.workunit.client.0.trial128.stdout: "total": 322122547200, 2026-02-17T12:10:52.418 INFO:tasks.workunit.client.0.trial128.stdout: "available": 322036199424, 2026-02-17T12:10:52.418 INFO:tasks.workunit.client.0.trial128.stdout: "internally_reserved": 0, 2026-02-17T12:10:52.418 INFO:tasks.workunit.client.0.trial128.stdout: "allocated": 1155072, 2026-02-17T12:10:52.418 INFO:tasks.workunit.client.0.trial128.stdout: "data_stored": 621525, 2026-02-17T12:10:52.418 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed": 0, 2026-02-17T12:10:52.418 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed_allocated": 0, 2026-02-17T12:10:52.418 INFO:tasks.workunit.client.0.trial128.stdout: "data_compressed_original": 0, 2026-02-17T12:10:52.418 INFO:tasks.workunit.client.0.trial128.stdout: "omap_allocated": 10388, 2026-02-17T12:10:52.418 INFO:tasks.workunit.client.0.trial128.stdout: "internal_metadata": 85120876 2026-02-17T12:10:52.418 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:10:52.418 INFO:tasks.workunit.client.0.trial128.stdout: "hb_peers": [], 2026-02-17T12:10:52.418 INFO:tasks.workunit.client.0.trial128.stdout: "snap_trim_queue_len": 0, 2026-02-17T12:10:52.419 INFO:tasks.workunit.client.0.trial128.stdout: "num_snap_trimming": 0, 2026-02-17T12:10:52.419 INFO:tasks.workunit.client.0.trial128.stdout: "num_shards_repaired": 26, 2026-02-17T12:10:52.419 INFO:tasks.workunit.client.0.trial128.stdout: "op_queue_age_hist": { 2026-02-17T12:10:52.419 INFO:tasks.workunit.client.0.trial128.stdout: "histogram": [], 2026-02-17T12:10:52.419 INFO:tasks.workunit.client.0.trial128.stdout: "upper_bound": 1 2026-02-17T12:10:52.419 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:10:52.419 INFO:tasks.workunit.client.0.trial128.stdout: "perf_stat": { 2026-02-17T12:10:52.419 INFO:tasks.workunit.client.0.trial128.stdout: "commit_latency_ms": 0, 2026-02-17T12:10:52.419 INFO:tasks.workunit.client.0.trial128.stdout: "apply_latency_ms": 0, 2026-02-17T12:10:52.419 INFO:tasks.workunit.client.0.trial128.stdout: "commit_latency_ns": 0, 2026-02-17T12:10:52.419 INFO:tasks.workunit.client.0.trial128.stdout: "apply_latency_ns": 0 2026-02-17T12:10:52.419 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:10:52.419 INFO:tasks.workunit.client.0.trial128.stdout: "alerts": [], 2026-02-17T12:10:52.420 INFO:tasks.workunit.client.0.trial128.stdout: "network_ping_times": [] 2026-02-17T12:10:52.420 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:10:52.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:793: TEST_repair_stats_ec: ceph pg dump --format=json-pretty 2026-02-17T12:10:52.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:793: TEST_repair_stats_ec: jq .pg_map.osd_stats_sum.num_shards_repaired 2026-02-17T12:10:52.561 INFO:tasks.workunit.client.0.trial128.stderr:dumped all 2026-02-17T12:10:52.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:793: TEST_repair_stats_ec: COUNT=26 2026-02-17T12:10:52.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:794: TEST_repair_stats_ec: test 26 = 26 2026-02-17T12:10:52.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T12:10:52.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:10:52.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:10:52.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:10:52.572 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:10:52.572 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:10:52.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:10:52.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:10:52.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:10:52.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:10:52.683 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:10:52.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:10:52.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:10:52.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:10:52.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:10:52.687 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:10:52.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:10:52.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:10:52.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:10:52.689 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:10:52.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:10:52.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:10:52.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:10:52.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:10:52.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:52.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:10:52.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:10:52.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:10:52.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:10:52.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T12:10:52.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T12:10:52.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T12:10:52.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T12:10:52.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:10:52.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:10:52.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:10:52.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:10:52.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:10:52.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:10:52.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:10:52.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:10:52.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:10:52.714 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:10:52.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:10:52.715 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:10:52.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:10:52.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:10:52.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:10:52.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:10:52.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:10:52.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:10:52.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:10:52.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:10:52.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:10:52.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:10:52.723 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:10:52.723 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:52.723 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:10:52.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:10:52.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:10:52.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:10:52.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T12:10:52.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:10:52.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:52.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:10:52.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T12:10:52.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:10:52.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:10:52.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:10:52.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T12:10:52.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_request_scrub_priority td/osd-scrub-repair 2026-02-17T12:10:52.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6197: TEST_request_scrub_priority: local dir=td/osd-scrub-repair 2026-02-17T12:10:52.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6198: TEST_request_scrub_priority: local poolname=psr_pool 2026-02-17T12:10:52.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6199: TEST_request_scrub_priority: local objname=POBJ 2026-02-17T12:10:52.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6200: TEST_request_scrub_priority: local OBJECTS=64 2026-02-17T12:10:52.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6201: TEST_request_scrub_priority: local PGS=8 2026-02-17T12:10:52.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6203: TEST_request_scrub_priority: run_mon td/osd-scrub-repair a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T12:10:52.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T12:10:52.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:10:52.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:10:52.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:10:52.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T12:10:52.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T12:10:52.751 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:10:52.751 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:10:52.751 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:10:52.751 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:10:52.751 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:52.752 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:10:52.752 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:10:52.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T12:10:52.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:10:52.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:10:52.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:10:52.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:10:52.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:10:52.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:10:52.784 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:10:52.784 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:10:52.784 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:10:52.785 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:10:52.785 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:52.785 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:10:52.785 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:10:52.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:10:52.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T12:10:52.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:10:52.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:10:52.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:10:52.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:10:52.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:10:52.842 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:10:52.842 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:10:52.842 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:10:52.843 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:10:52.843 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:52.843 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:10:52.843 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:10:52.843 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:10:52.843 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T12:10:52.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6204: TEST_request_scrub_priority: run_mgr td/osd-scrub-repair x 2026-02-17T12:10:52.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T12:10:52.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:10:52.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:10:52.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:10:52.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T12:10:52.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:10:52.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:10:52.990 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:10:52.990 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:10:52.990 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:10:52.990 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:52.990 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:10:52.991 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:10:52.991 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:10:52.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:10:53.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6205: TEST_request_scrub_priority: local 'ceph_osd_args=--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 ' 2026-02-17T12:10:53.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6206: TEST_request_scrub_priority: ceph_osd_args+=--osd_scrub_backoff_ratio=0 2026-02-17T12:10:53.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6207: TEST_request_scrub_priority: run_osd td/osd-scrub-repair 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-17T12:10:53.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:10:53.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:10:53.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:10:53.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:10:53.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:10:53.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:10:53.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:10:53.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:10:53.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:10:53.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:10:53.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:10:53.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:10:53.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:10:53.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:10:53.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:10:53.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:10:53.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:10:53.015 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:10:53.015 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:10:53.015 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:10:53.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:10:53.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:10:53.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:10:53.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:10:53.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:10:53.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:10:53.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:10:53.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:10:53.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:10:53.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:10:53.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:10:53.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:10:53.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0' 2026-02-17T12:10:53.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:10:53.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:10:53.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=bf9f8217-141b-4343-9329-3c3f6755b133 2026-02-17T12:10:53.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 bf9f8217-141b-4343-9329-3c3f6755b133' 2026-02-17T12:10:53.018 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 bf9f8217-141b-4343-9329-3c3f6755b133 2026-02-17T12:10:53.018 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:10:53.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDNWpRpVnMzARAAtN0eNqoffA4w0iqWunF/rQ== 2026-02-17T12:10:53.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDNWpRpVnMzARAAtN0eNqoffA4w0iqWunF/rQ=="}' 2026-02-17T12:10:53.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new bf9f8217-141b-4343-9329-3c3f6755b133 -i td/osd-scrub-repair/0/new.json 2026-02-17T12:10:53.117 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:10:53.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T12:10:53.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --mkfs --key AQDNWpRpVnMzARAAtN0eNqoffA4w0iqWunF/rQ== --osd-uuid bf9f8217-141b-4343-9329-3c3f6755b133 2026-02-17T12:10:53.144 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:53.134+0000 7effc5fdb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:53.145 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:53.134+0000 7effc5fdb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:53.146 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:53.134+0000 7effc5fdb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:53.147 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:53.138+0000 7effc5fdb8c0 -1 bdev(0x55c0b0dd1c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:10:53.147 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:53.138+0000 7effc5fdb8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T12:10:55.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T12:10:55.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:10:55.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:10:55.965 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:10:55.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:10:56.158 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:10:56.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:10:56.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 2026-02-17T12:10:56.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:10:56.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:10:56.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:10:56.183 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:56.174+0000 7f63959408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:56.185 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:56.174+0000 7f63959408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:56.186 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:56.174+0000 7f63959408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:10:56.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:10:56.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:10:56.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:10:56.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:10:56.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:10:56.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:56.314 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:10:56.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:10:56.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:56.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:10:56.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:57.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:57.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:57.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:10:57.471 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:10:57.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:57.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:10:57.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:57.672 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:10:57.662+0000 7f63959408c0 -1 Falling back to public interface 2026-02-17T12:10:58.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:58.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:58.623 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:10:58.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:10:58.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:58.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:10:58.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:10:59.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:10:59.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:10:59.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:10:59.773 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:10:59.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:10:59.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:10:59.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:00.654 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:11:00.642+0000 7f63959408c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:11:00.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:00.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:00.920 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:11:00.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:11:00.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:00.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:11:01.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:02.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:02.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:02.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:11:02.082 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:11:02.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:02.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:11:02.233 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/759374286,v1:127.0.0.1:6803/759374286] [v2:127.0.0.1:6804/759374286,v1:127.0.0.1:6805/759374286] exists,up bf9f8217-141b-4343-9329-3c3f6755b133 2026-02-17T12:11:02.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:11:02.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:11:02.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:11:02.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6209: TEST_request_scrub_priority: create_pool psr_pool 8 8 2026-02-17T12:11:02.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool 8 8 2026-02-17T12:11:02.432 INFO:tasks.workunit.client.0.trial128.stderr:pool 'psr_pool' created 2026-02-17T12:11:02.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:11:03.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6210: TEST_request_scrub_priority: wait_for_clean 2026-02-17T12:11:03.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:11:03.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:11:03.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:11:03.444 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:11:03.445 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:11:03.445 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:11:03.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:11:03.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:11:03.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:11:03.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:11:03.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:11:03.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:11:03.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:11:03.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:11:03.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:11:03.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T12:11:03.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:11:03.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:11:03.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:11:03.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836482 2026-02-17T12:11:03.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836482 2026-02-17T12:11:03.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836482' 2026-02-17T12:11:03.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:11:03.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836482 2026-02-17T12:11:03.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:11:03.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:11:03.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836482 2026-02-17T12:11:03.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:11:03.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836482 2026-02-17T12:11:03.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836482' 2026-02-17T12:11:03.798 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836482 2026-02-17T12:11:03.798 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:11:03.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836481 -lt 21474836482 2026-02-17T12:11:03.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:11:04.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:11:04.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:11:05.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836481 -lt 21474836482 2026-02-17T12:11:05.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:11:06.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:11:06.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:11:06.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836482 2026-02-17T12:11:06.267 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:11:06.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:11:06.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:11:06.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 8 == 0 2026-02-17T12:11:06.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:11:06.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:11:06.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:11:06.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:11:06.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:11:06.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:11:06.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:11:06.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=8 2026-02-17T12:11:06.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:11:06.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:11:06.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:11:06.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 8 = 8 2026-02-17T12:11:06.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:11:06.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:11:06.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6212: TEST_request_scrub_priority: local osd=0 2026-02-17T12:11:06.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6213: TEST_request_scrub_priority: add_something td/osd-scrub-repair psr_pool POBJ noscrub 2026-02-17T12:11:06.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T12:11:06.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=psr_pool 2026-02-17T12:11:06.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=POBJ 2026-02-17T12:11:06.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T12:11:06.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T12:11:06.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T12:11:06.978 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:11:06.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T12:11:07.183 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:11:07.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T12:11:07.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T12:11:07.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool psr_pool put POBJ td/osd-scrub-repair/ORIGINAL 2026-02-17T12:11:07.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6214: TEST_request_scrub_priority: get_primary psr_pool POBJ 2026-02-17T12:11:07.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=psr_pool 2026-02-17T12:11:07.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=POBJ 2026-02-17T12:11:07.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map psr_pool POBJ 2026-02-17T12:11:07.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:11:07.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6214: TEST_request_scrub_priority: local primary=0 2026-02-17T12:11:07.374 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6215: TEST_request_scrub_priority: get_pg psr_pool POBJ 2026-02-17T12:11:07.374 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=psr_pool 2026-02-17T12:11:07.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=POBJ 2026-02-17T12:11:07.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map psr_pool POBJ 2026-02-17T12:11:07.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T12:11:07.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6215: TEST_request_scrub_priority: local pg=1.1 2026-02-17T12:11:07.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6216: TEST_request_scrub_priority: ceph osd dump 2026-02-17T12:11:07.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6216: TEST_request_scrub_priority: grep '^pool.*['\'']psr_pool['\'']' 2026-02-17T12:11:07.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6216: TEST_request_scrub_priority: awk '{ print $2 }' 2026-02-17T12:11:07.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6216: TEST_request_scrub_priority: poolid=1 2026-02-17T12:11:07.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6218: TEST_request_scrub_priority: local otherpgs 2026-02-17T12:11:07.675 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: expr 8 - 1 2026-02-17T12:11:07.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: seq 0 7 2026-02-17T12:11:07.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-17T12:11:07.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6221: TEST_request_scrub_priority: opg=1.0 2026-02-17T12:11:07.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6222: TEST_request_scrub_priority: '[' 1.0 = 1.1 ']' 2026-02-17T12:11:07.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6225: TEST_request_scrub_priority: otherpgs='1.0 ' 2026-02-17T12:11:07.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-17T12:11:07.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T12:11:07.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:07.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:07.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T12:11:07.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: local other_last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:07.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6228: TEST_request_scrub_priority: ceph tell 1.0 schedule-scrub 1.0 2026-02-17T12:11:07.896 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:11:07.896 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:11:07.896 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:11:07.896 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-10T12:09:27.890719+0000" 2026-02-17T12:11:07.896 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:11:07.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-17T12:11:07.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6221: TEST_request_scrub_priority: opg=1.1 2026-02-17T12:11:07.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6222: TEST_request_scrub_priority: '[' 1.1 = 1.1 ']' 2026-02-17T12:11:07.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6223: TEST_request_scrub_priority: continue 2026-02-17T12:11:07.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-17T12:11:07.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6221: TEST_request_scrub_priority: opg=1.2 2026-02-17T12:11:07.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6222: TEST_request_scrub_priority: '[' 1.2 = 1.1 ']' 2026-02-17T12:11:07.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6225: TEST_request_scrub_priority: otherpgs='1.0 1.2 ' 2026-02-17T12:11:07.908 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-17T12:11:07.908 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T12:11:07.908 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:07.908 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:07.908 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T12:11:08.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: local other_last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:08.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6228: TEST_request_scrub_priority: ceph tell 1.2 schedule-scrub 1.2 2026-02-17T12:11:08.122 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:11:08.123 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:11:08.123 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:11:08.123 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-10T12:09:28.117605+0000" 2026-02-17T12:11:08.123 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:11:08.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-17T12:11:08.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6221: TEST_request_scrub_priority: opg=1.3 2026-02-17T12:11:08.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6222: TEST_request_scrub_priority: '[' 1.3 = 1.1 ']' 2026-02-17T12:11:08.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6225: TEST_request_scrub_priority: otherpgs='1.0 1.2 1.3 ' 2026-02-17T12:11:08.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-17T12:11:08.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T12:11:08.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:08.132 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:08.132 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T12:11:08.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: local other_last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:08.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6228: TEST_request_scrub_priority: ceph tell 1.3 schedule-scrub 1.3 2026-02-17T12:11:08.350 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:11:08.350 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:11:08.350 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:11:08.350 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-10T12:09:28.344755+0000" 2026-02-17T12:11:08.350 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:11:08.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-17T12:11:08.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6221: TEST_request_scrub_priority: opg=1.4 2026-02-17T12:11:08.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6222: TEST_request_scrub_priority: '[' 1.4 = 1.1 ']' 2026-02-17T12:11:08.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6225: TEST_request_scrub_priority: otherpgs='1.0 1.2 1.3 1.4 ' 2026-02-17T12:11:08.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-17T12:11:08.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T12:11:08.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:08.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:08.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T12:11:08.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: local other_last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:08.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6228: TEST_request_scrub_priority: ceph tell 1.4 schedule-scrub 1.4 2026-02-17T12:11:08.584 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:11:08.584 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:11:08.584 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:11:08.584 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-10T12:09:28.578995+0000" 2026-02-17T12:11:08.584 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:11:08.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-17T12:11:08.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6221: TEST_request_scrub_priority: opg=1.5 2026-02-17T12:11:08.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6222: TEST_request_scrub_priority: '[' 1.5 = 1.1 ']' 2026-02-17T12:11:08.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6225: TEST_request_scrub_priority: otherpgs='1.0 1.2 1.3 1.4 1.5 ' 2026-02-17T12:11:08.592 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-17T12:11:08.592 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T12:11:08.592 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:08.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:08.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T12:11:08.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: local other_last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:08.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6228: TEST_request_scrub_priority: ceph tell 1.5 schedule-scrub 1.5 2026-02-17T12:11:08.814 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:11:08.814 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:11:08.814 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:11:08.814 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-10T12:09:28.808987+0000" 2026-02-17T12:11:08.814 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:11:08.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-17T12:11:08.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6221: TEST_request_scrub_priority: opg=1.6 2026-02-17T12:11:08.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6222: TEST_request_scrub_priority: '[' 1.6 = 1.1 ']' 2026-02-17T12:11:08.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6225: TEST_request_scrub_priority: otherpgs='1.0 1.2 1.3 1.4 1.5 1.6 ' 2026-02-17T12:11:08.821 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-17T12:11:08.821 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T12:11:08.821 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:08.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:08.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T12:11:08.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: local other_last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:08.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6228: TEST_request_scrub_priority: ceph tell 1.6 schedule-scrub 1.6 2026-02-17T12:11:09.039 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:11:09.039 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:11:09.039 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:11:09.039 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-10T12:09:29.033703+0000" 2026-02-17T12:11:09.039 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:11:09.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6219: TEST_request_scrub_priority: for i in $(seq 0 $(expr $PGS - 1)) 2026-02-17T12:11:09.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6221: TEST_request_scrub_priority: opg=1.7 2026-02-17T12:11:09.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6222: TEST_request_scrub_priority: '[' 1.7 = 1.1 ']' 2026-02-17T12:11:09.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6225: TEST_request_scrub_priority: otherpgs='1.0 1.2 1.3 1.4 1.5 1.6 1.7 ' 2026-02-17T12:11:09.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-17T12:11:09.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T12:11:09.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:09.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:09.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T12:11:09.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6226: TEST_request_scrub_priority: local other_last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:09.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6228: TEST_request_scrub_priority: ceph tell 1.7 schedule-scrub 1.7 2026-02-17T12:11:09.265 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:11:09.266 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:11:09.266 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:11:09.266 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-10T12:09:29.260519+0000" 2026-02-17T12:11:09.266 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:11:09.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6231: TEST_request_scrub_priority: sleep 15 2026-02-17T12:11:24.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6232: TEST_request_scrub_priority: flush_pg_stats 2026-02-17T12:11:24.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:11:24.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:11:24.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T12:11:24.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:11:24.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:11:24.430 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:11:24.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T12:11:24.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T12:11:24.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T12:11:24.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:11:24.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T12:11:24.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:11:24.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:11:24.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T12:11:24.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:11:24.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T12:11:24.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T12:11:24.505 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836487 2026-02-17T12:11:24.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:11:24.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-17T12:11:24.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:11:25.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:11:25.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:11:25.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T12:11:25.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6235: TEST_request_scrub_priority: get_last_scrub_stamp 1.1 2026-02-17T12:11:25.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T12:11:25.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:25.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:25.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T12:11:25.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6235: TEST_request_scrub_priority: local last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:25.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6236: TEST_request_scrub_priority: ceph tell 1.1 scrub 2026-02-17T12:11:26.030 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:11:26.030 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:11:26.030 INFO:tasks.workunit.client.0.trial128.stdout: "must": true, 2026-02-17T12:11:26.030 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "0.000000" 2026-02-17T12:11:26.030 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:11:26.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6238: TEST_request_scrub_priority: ceph osd unset noscrub 2026-02-17T12:11:26.240 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is unset 2026-02-17T12:11:26.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6239: TEST_request_scrub_priority: ceph osd unset nodeep-scrub 2026-02-17T12:11:26.444 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is unset 2026-02-17T12:11:26.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6241: TEST_request_scrub_priority: wait_for_scrub 1.1 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:26.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.1 2026-02-17T12:11:26.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:26.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:11:26.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:11:26.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:26.457 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.1 last_scrub_stamp 2026-02-17T12:11:26.457 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T12:11:26.458 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:26.458 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:26.458 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T12:11:26.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:11:02.425486+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:26.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:11:27.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:11:27.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:27.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.1 last_scrub_stamp 2026-02-17T12:11:27.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T12:11:27.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:27.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T12:11:27.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:27.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:11:26.668558+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:27.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:11:27.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6243: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-17T12:11:27.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6245: TEST_request_scrub_priority: wait_for_scrub 1.0 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:27.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T12:11:27.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:27.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:11:27.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:11:27.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:27.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:11:27.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:11:27.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:27.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:27.764 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:11:27.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:09:27.890719+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:27.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:11:28.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:11:28.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:28.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:11:28.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:11:28.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:28.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:11:28.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:29.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:09:27.890719+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:29.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:11:30.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:11:30.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:30.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:11:30.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:11:30.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:30.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:30.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:11:30.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:09:27.890719+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:30.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:11:31.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:11:31.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:31.221 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:11:31.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:11:31.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:31.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:11:31.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:31.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:09:27.890719+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:31.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:11:32.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:11:32.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:32.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:11:32.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:11:32.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:32.376 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:11:32.376 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:32.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:09:27.890719+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:32.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:11:33.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:11:33.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:33.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:11:33.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:11:33.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:33.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:11:33.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:33.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:11:31.609628+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:33.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:11:33.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6243: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-17T12:11:33.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6245: TEST_request_scrub_priority: wait_for_scrub 1.2 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:33.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.2 2026-02-17T12:11:33.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:33.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:11:33.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:11:33.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:33.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.2 last_scrub_stamp 2026-02-17T12:11:33.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.2 2026-02-17T12:11:33.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:33.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:33.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.2") | .last_scrub_stamp' 2026-02-17T12:11:33.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:09:28.117605+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:33.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:11:34.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:11:34.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:34.832 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.2 last_scrub_stamp 2026-02-17T12:11:34.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.2 2026-02-17T12:11:34.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:34.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.2") | .last_scrub_stamp' 2026-02-17T12:11:34.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:34.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:09:28.117605+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:34.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:11:35.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:11:35.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:35.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.2 last_scrub_stamp 2026-02-17T12:11:35.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.2 2026-02-17T12:11:35.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:35.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.2") | .last_scrub_stamp' 2026-02-17T12:11:35.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:36.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:09:28.117605+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:36.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:11:37.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:11:37.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:37.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.2 last_scrub_stamp 2026-02-17T12:11:37.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.2 2026-02-17T12:11:37.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:37.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.2") | .last_scrub_stamp' 2026-02-17T12:11:37.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:37.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:11:32.596825+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:37.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:11:37.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6243: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-17T12:11:37.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6245: TEST_request_scrub_priority: wait_for_scrub 1.3 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:37.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.3 2026-02-17T12:11:37.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:37.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:11:37.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:11:37.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:37.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.3 last_scrub_stamp 2026-02-17T12:11:37.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T12:11:37.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:37.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:37.294 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T12:11:37.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:11:33.581810+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:37.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:11:37.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6243: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-17T12:11:37.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6245: TEST_request_scrub_priority: wait_for_scrub 1.4 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:37.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.4 2026-02-17T12:11:37.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:37.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:11:37.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:11:37.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:37.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.4 last_scrub_stamp 2026-02-17T12:11:37.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.4 2026-02-17T12:11:37.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:37.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:37.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.4") | .last_scrub_stamp' 2026-02-17T12:11:37.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:11:34.545541+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:37.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:11:37.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6243: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-17T12:11:37.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6245: TEST_request_scrub_priority: wait_for_scrub 1.5 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:37.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.5 2026-02-17T12:11:37.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:37.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:11:37.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:11:37.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:37.587 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.5 last_scrub_stamp 2026-02-17T12:11:37.587 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.5 2026-02-17T12:11:37.587 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:37.587 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:37.587 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.5") | .last_scrub_stamp' 2026-02-17T12:11:37.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:11:35.498796+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:37.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:11:37.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6243: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-17T12:11:37.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6245: TEST_request_scrub_priority: wait_for_scrub 1.6 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:37.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.6 2026-02-17T12:11:37.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:37.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:11:37.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:11:37.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:37.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.6 last_scrub_stamp 2026-02-17T12:11:37.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.6 2026-02-17T12:11:37.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:37.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:37.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.6") | .last_scrub_stamp' 2026-02-17T12:11:37.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:11:36.463137+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:37.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:11:37.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6243: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-17T12:11:37.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6245: TEST_request_scrub_priority: wait_for_scrub 1.7 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:37.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.7 2026-02-17T12:11:37.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:37.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:11:37.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:11:37.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:37.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.7 last_scrub_stamp 2026-02-17T12:11:37.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.7 2026-02-17T12:11:37.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:37.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:37.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.7") | .last_scrub_stamp' 2026-02-17T12:11:38.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:09:29.260519+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:38.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:11:39.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:11:39.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:39.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.7 last_scrub_stamp 2026-02-17T12:11:39.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.7 2026-02-17T12:11:39.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:39.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.7") | .last_scrub_stamp' 2026-02-17T12:11:39.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:39.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:09:29.260519+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:39.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:11:40.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:11:40.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:40.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.7 last_scrub_stamp 2026-02-17T12:11:40.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.7 2026-02-17T12:11:40.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:40.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:40.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.7") | .last_scrub_stamp' 2026-02-17T12:11:40.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:09:29.260519+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:40.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:11:41.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:11:41.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:41.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.7 last_scrub_stamp 2026-02-17T12:11:41.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.7 2026-02-17T12:11:41.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:41.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.7") | .last_scrub_stamp' 2026-02-17T12:11:41.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:41.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:09:29.260519+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:41.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:11:42.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:11:42.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:42.492 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.7 last_scrub_stamp 2026-02-17T12:11:42.492 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.7 2026-02-17T12:11:42.492 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:42.493 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.7") | .last_scrub_stamp' 2026-02-17T12:11:42.493 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:42.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:09:29.260519+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:42.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:11:43.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:11:43.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:43.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.7 last_scrub_stamp 2026-02-17T12:11:43.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.7 2026-02-17T12:11:43.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:43.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.7") | .last_scrub_stamp' 2026-02-17T12:11:43.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:43.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:11:37.413777+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:43.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:11:43.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6243: TEST_request_scrub_priority: for opg in $otherpgs $pg 2026-02-17T12:11:43.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6245: TEST_request_scrub_priority: wait_for_scrub 1.1 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:43.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.1 2026-02-17T12:11:43.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:43.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:11:43.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:11:43.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:11:43.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.1 last_scrub_stamp 2026-02-17T12:11:43.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.1 2026-02-17T12:11:43.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:11:43.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:11:43.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.1") | .last_scrub_stamp' 2026-02-17T12:11:43.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:11:26.668558+0000 '>' 2026-02-17T12:11:02.425486+0000 2026-02-17T12:11:43.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:11:43.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6249: TEST_request_scrub_priority: grep 'log_channel.*scrub ok' td/osd-scrub-repair/osd.0.log 2026-02-17T12:11:43.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6249: TEST_request_scrub_priority: grep -v purged_snaps 2026-02-17T12:11:43.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6249: TEST_request_scrub_priority: head -1 2026-02-17T12:11:43.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6249: TEST_request_scrub_priority: sed 's/.*[[]DBG[]]//' 2026-02-17T12:11:43.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:6249: TEST_request_scrub_priority: grep -q 1.1 2026-02-17T12:11:43.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T12:11:43.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:11:43.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:11:43.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:11:43.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:11:43.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:11:43.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:11:43.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:11:43.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:11:44.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:11:44.057 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:11:44.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:11:44.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:11:44.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:11:44.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:11:44.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:11:44.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:11:44.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:11:44.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:11:44.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:11:44.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:11:44.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:11:44.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:11:44.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:11:44.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:44.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:11:44.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:11:44.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:11:44.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:11:44.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T12:11:44.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T12:11:44.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T12:11:44.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T12:11:44.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:11:44.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:11:44.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:11:44.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:11:44.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:11:44.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:11:44.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:11:44.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:11:44.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:11:44.078 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:11:44.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:11:44.079 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:11:44.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:11:44.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:11:44.082 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:11:44.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:11:44.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:11:44.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:11:44.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:11:44.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:11:44.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:11:44.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:11:44.087 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:11:44.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:44.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:11:44.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:11:44.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:11:44.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:11:44.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T12:11:44.091 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:11:44.091 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:44.091 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:11:44.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T12:11:44.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:11:44.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:11:44.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:11:44.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T12:11:44.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_scrub_warning td/osd-scrub-repair 2026-02-17T12:11:44.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5842: TEST_scrub_warning: local dir=td/osd-scrub-repair 2026-02-17T12:11:44.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5843: TEST_scrub_warning: local poolname=psr_pool 2026-02-17T12:11:44.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5844: TEST_scrub_warning: local objname=POBJ 2026-02-17T12:11:44.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5845: TEST_scrub_warning: local scrubs=5 2026-02-17T12:11:44.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5846: TEST_scrub_warning: local deep_scrubs=5 2026-02-17T12:11:44.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5847: TEST_scrub_warning: local i1_day=86400 2026-02-17T12:11:44.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5848: TEST_scrub_warning: calc 86400 '*' 7 2026-02-17T12:11:44.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1589: calc: awk 'BEGIN{print 86400 * 7}' 2026-02-17T12:11:44.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5848: TEST_scrub_warning: local i7_days=604800 2026-02-17T12:11:44.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5849: TEST_scrub_warning: calc 86400 '*' 14 2026-02-17T12:11:44.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1589: calc: awk 'BEGIN{print 86400 * 14}' 2026-02-17T12:11:44.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5849: TEST_scrub_warning: local i14_days=1209600 2026-02-17T12:11:44.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5850: TEST_scrub_warning: local overdue=0.5 2026-02-17T12:11:44.099 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5851: TEST_scrub_warning: calc 604800 + 86400 + '(' 604800 '*' 0.5 ')' 2026-02-17T12:11:44.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1589: calc: awk 'BEGIN{print 604800 + 86400 + ( 604800 * 0.5 )}' 2026-02-17T12:11:44.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5851: TEST_scrub_warning: local conf_overdue_seconds=993600 2026-02-17T12:11:44.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5852: TEST_scrub_warning: calc 1209600 + 86400 + '(' 1209600 '*' 0.5 ')' 2026-02-17T12:11:44.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1589: calc: awk 'BEGIN{print 1209600 + 86400 + ( 1209600 * 0.5 )}' 2026-02-17T12:11:44.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5852: TEST_scrub_warning: local pool_overdue_seconds=1900800 2026-02-17T12:11:44.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5854: TEST_scrub_warning: run_mon td/osd-scrub-repair a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T12:11:44.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T12:11:44.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:11:44.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:11:44.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:11:44.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T12:11:44.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T12:11:44.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:11:44.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:11:44.127 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:11:44.127 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:11:44.127 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:44.127 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:11:44.127 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:11:44.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-17T12:11:44.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:11:44.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:11:44.156 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:11:44.156 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:11:44.156 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:11:44.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:11:44.157 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:11:44.157 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:11:44.157 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:11:44.157 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:11:44.157 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:44.157 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:11:44.158 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:11:44.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:11:44.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T12:11:44.216 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:11:44.216 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:11:44.216 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:11:44.216 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:11:44.216 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:11:44.217 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:11:44.217 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:11:44.217 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:11:44.217 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:11:44.218 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:44.218 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:11:44.218 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:11:44.218 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:11:44.218 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T12:11:44.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5855: TEST_scrub_warning: run_mgr td/osd-scrub-repair x --mon_warn_pg_not_scrubbed_ratio=0.5 --mon_warn_pg_not_deep_scrubbed_ratio=0.5 2026-02-17T12:11:44.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T12:11:44.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:11:44.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:11:44.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:11:44.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T12:11:44.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:11:44.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:11:44.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:11:44.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:11:44.370 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:11:44.370 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:44.370 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:11:44.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:11:44.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:11:44.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mon_warn_pg_not_scrubbed_ratio=0.5 --mon_warn_pg_not_deep_scrubbed_ratio=0.5 2026-02-17T12:11:44.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5856: TEST_scrub_warning: run_osd td/osd-scrub-repair 0 --osd_scrub_backoff_ratio=0 2026-02-17T12:11:44.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:11:44.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:11:44.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:11:44.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:11:44.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:11:44.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:11:44.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:11:44.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:11:44.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:11:44.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:11:44.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:11:44.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:11:44.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:11:44.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:11:44.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:11:44.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:11:44.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:11:44.396 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:11:44.396 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:11:44.396 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:11:44.396 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:11:44.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:11:44.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:11:44.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:11:44.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:11:44.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:11:44.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:11:44.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:11:44.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:11:44.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:11:44.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:11:44.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:11:44.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+=--osd_scrub_backoff_ratio=0 2026-02-17T12:11:44.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:11:44.398 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:11:44.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=05e1ec43-100c-4937-9a75-cd3bfbe4e84d 2026-02-17T12:11:44.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 05e1ec43-100c-4937-9a75-cd3bfbe4e84d' 2026-02-17T12:11:44.398 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 05e1ec43-100c-4937-9a75-cd3bfbe4e84d 2026-02-17T12:11:44.398 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:11:44.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAAW5Rp1x3WFxAAOcOg8rNSekrj03GMG3D/2g== 2026-02-17T12:11:44.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAAW5Rp1x3WFxAAOcOg8rNSekrj03GMG3D/2g=="}' 2026-02-17T12:11:44.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 05e1ec43-100c-4937-9a75-cd3bfbe4e84d -i td/osd-scrub-repair/0/new.json 2026-02-17T12:11:44.492 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:11:44.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T12:11:44.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_backoff_ratio=0 --mkfs --key AQAAW5Rp1x3WFxAAOcOg8rNSekrj03GMG3D/2g== --osd-uuid 05e1ec43-100c-4937-9a75-cd3bfbe4e84d 2026-02-17T12:11:44.517 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:11:44.509+0000 7f0deeb4b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:44.518 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:11:44.509+0000 7f0deeb4b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:44.519 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:11:44.513+0000 7f0deeb4b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:44.519 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:11:44.513+0000 7f0deeb4b8c0 -1 bdev(0x5631af7efc00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:11:44.520 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:11:44.513+0000 7f0deeb4b8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T12:11:46.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T12:11:46.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:11:46.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:11:46.822 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:11:46.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:11:47.019 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:11:47.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:11:47.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_backoff_ratio=0 2026-02-17T12:11:47.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:11:47.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:11:47.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:11:47.043 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:11:47.032+0000 7f37d80858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:47.044 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:11:47.036+0000 7f37d80858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:47.046 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:11:47.036+0000 7f37d80858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:11:47.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:11:47.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:11:47.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:11:47.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:11:47.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:11:47.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:47.168 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:11:47.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:11:47.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:47.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:11:47.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:48.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:48.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:48.326 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:11:48.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:11:48.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:48.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:11:48.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:48.773 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:11:48.764+0000 7f37d80858c0 -1 Falling back to public interface 2026-02-17T12:11:49.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:49.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:49.483 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:11:49.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:11:49.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:49.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:11:49.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:50.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:50.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:50.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:11:50.639 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:11:50.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:50.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:11:50.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:51.785 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:11:51.776+0000 7f37d80858c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:11:51.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:51.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:51.797 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:11:51.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:11:51.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:51.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:11:51.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:11:52.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:11:52.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:11:52.949 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:11:52.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:11:52.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:11:52.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:11:53.102 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2353880816,v1:127.0.0.1:6803/2353880816] [v2:127.0.0.1:6804/2353880816,v1:127.0.0.1:6805/2353880816] exists,up 05e1ec43-100c-4937-9a75-cd3bfbe4e84d 2026-02-17T12:11:53.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:11:53.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:11:53.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:11:53.104 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: expr 5 + 5 2026-02-17T12:11:53.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: seq 1 10 2026-02-17T12:11:53.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T12:11:53.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-1 1 1 2026-02-17T12:11:53.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-1 1 1 2026-02-17T12:11:53.319 INFO:tasks.workunit.client.0.trial128.stderr:pool 'psr_pool-1' created 2026-02-17T12:11:53.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:11:54.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T12:11:54.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:11:54.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:11:54.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:11:54.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:11:54.333 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:11:54.333 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:11:54.333 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:11:54.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:11:54.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:11:54.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:11:54.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:11:54.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:11:54.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:11:54.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:11:54.456 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:11:54.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T12:11:54.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:11:54.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:11:54.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:11:54.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836482 2026-02-17T12:11:54.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836482 2026-02-17T12:11:54.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836482' 2026-02-17T12:11:54.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:11:54.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836482 2026-02-17T12:11:54.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:11:54.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:11:54.681 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836482 2026-02-17T12:11:54.681 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:11:54.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836482 2026-02-17T12:11:54.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836482' 2026-02-17T12:11:54.683 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836482 2026-02-17T12:11:54.683 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:11:54.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836481 -lt 21474836482 2026-02-17T12:11:54.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:11:55.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:11:55.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:11:56.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836481 -lt 21474836482 2026-02-17T12:11:56.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:11:57.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:11:57.004 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:11:57.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836482 2026-02-17T12:11:57.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:11:57.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:11:57.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:11:57.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:11:57.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:11:57.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:11:57.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:11:57.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:11:57.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:11:57.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:11:57.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:11:57.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:11:57.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:11:57.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:11:57.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:11:57.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:11:57.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:11:57.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:11:57.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 1 = 1 ']' 2026-02-17T12:11:57.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5864: TEST_scrub_warning: ceph osd pool set psr_pool-1 scrub_max_interval 1209600 2026-02-17T12:11:57.858 INFO:tasks.workunit.client.0.trial128.stderr:set pool 1 scrub_max_interval to 1209600 2026-02-17T12:11:57.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T12:11:57.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 1 = 6 ']' 2026-02-17T12:11:57.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T12:11:57.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-2 1 1 2026-02-17T12:11:57.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-2 1 1 2026-02-17T12:11:58.064 INFO:tasks.workunit.client.0.trial128.stderr:pool 'psr_pool-2' created 2026-02-17T12:11:58.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:11:59.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T12:11:59.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:11:59.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:11:59.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:11:59.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:11:59.077 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:11:59.077 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:11:59.077 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:11:59.078 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:11:59.078 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:11:59.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:11:59.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:11:59.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:11:59.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:11:59.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:11:59.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:11:59.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T12:11:59.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:11:59.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:11:59.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:11:59.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T12:11:59.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T12:11:59.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T12:11:59.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:11:59.430 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T12:11:59.430 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:11:59.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:11:59.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T12:11:59.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:11:59.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T12:11:59.434 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836484 2026-02-17T12:11:59.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T12:11:59.434 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:11:59.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836483 -lt 21474836484 2026-02-17T12:11:59.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:00.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:12:00.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:00.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T12:12:00.759 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:12:00.760 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:00.760 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:00.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 2 == 0 2026-02-17T12:12:00.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:12:00.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:12:00.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:12:00.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:12:00.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:12:00.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:12:00.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:12:01.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=2 2026-02-17T12:12:01.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:12:01.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:01.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:01.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 2 = 2 2026-02-17T12:12:01.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:12:01.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:12:01.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 2 = 1 ']' 2026-02-17T12:12:01.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T12:12:01.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 2 = 6 ']' 2026-02-17T12:12:01.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T12:12:01.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-3 1 1 2026-02-17T12:12:01.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-3 1 1 2026-02-17T12:12:01.525 INFO:tasks.workunit.client.0.trial128.stderr:pool 'psr_pool-3' created 2026-02-17T12:12:01.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:12:02.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T12:12:02.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:12:02.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:12:02.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:12:02.541 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:12:02.542 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:12:02.542 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:12:02.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:12:02.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:12:02.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:12:02.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:12:02.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:12:02.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:12:02.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:12:02.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:12:02.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:12:02.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T12:12:02.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:12:02.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:02.807 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:12:02.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T12:12:02.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T12:12:02.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T12:12:02.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:02.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T12:12:02.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:02.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:12:02.880 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T12:12:02.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:02.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T12:12:02.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T12:12:02.883 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836485 2026-02-17T12:12:02.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:03.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T12:12:03.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:04.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:12:04.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:04.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T12:12:04.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:05.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:12:05.188 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:05.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-17T12:12:05.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:12:05.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:05.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:05.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 3 == 0 2026-02-17T12:12:05.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:12:05.520 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:12:05.520 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:12:05.521 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:12:05.521 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:12:05.521 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:12:05.521 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:12:05.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=3 2026-02-17T12:12:05.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:12:05.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:05.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:05.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 3 = 3 2026-02-17T12:12:05.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:12:05.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:12:05.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 3 = 1 ']' 2026-02-17T12:12:05.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T12:12:05.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 3 = 6 ']' 2026-02-17T12:12:05.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T12:12:05.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-4 1 1 2026-02-17T12:12:05.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-4 1 1 2026-02-17T12:12:06.046 INFO:tasks.workunit.client.0.trial128.stderr:pool 'psr_pool-4' created 2026-02-17T12:12:06.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:12:07.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T12:12:07.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:12:07.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:12:07.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:12:07.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:12:07.059 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:12:07.059 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:12:07.060 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:12:07.060 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:12:07.060 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:12:07.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:12:07.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:12:07.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:12:07.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:12:07.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:12:07.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:12:07.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T12:12:07.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:12:07.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:07.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:12:07.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T12:12:07.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T12:12:07.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T12:12:07.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:07.411 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T12:12:07.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:07.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:12:07.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T12:12:07.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:07.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T12:12:07.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T12:12:07.416 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836487 2026-02-17T12:12:07.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:07.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-17T12:12:07.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:08.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:12:08.572 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:08.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836487 2026-02-17T12:12:08.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:12:08.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:08.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:08.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T12:12:08.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:12:08.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:12:08.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:12:08.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:12:08.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:12:08.920 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:12:08.920 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:12:09.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T12:12:09.074 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:12:09.074 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:09.074 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:09.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T12:12:09.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:12:09.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:12:09.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 4 = 1 ']' 2026-02-17T12:12:09.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T12:12:09.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 4 = 6 ']' 2026-02-17T12:12:09.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T12:12:09.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-5 1 1 2026-02-17T12:12:09.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-5 1 1 2026-02-17T12:12:09.509 INFO:tasks.workunit.client.0.trial128.stderr:pool 'psr_pool-5' created 2026-02-17T12:12:09.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:12:10.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T12:12:10.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:12:10.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:12:10.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:12:10.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:12:10.523 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:12:10.523 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:12:10.524 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:12:10.524 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:12:10.524 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:12:10.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:12:10.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:12:10.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:12:10.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:12:10.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:12:10.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:12:10.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T12:12:10.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:12:10.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:10.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:12:10.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-17T12:12:10.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-17T12:12:10.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-17T12:12:10.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:10.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-17T12:12:10.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:10.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:12:10.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-17T12:12:10.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:10.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-17T12:12:10.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-17T12:12:10.871 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836489 2026-02-17T12:12:10.871 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:11.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-17T12:12:11.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:12.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:12:12.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:12.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836489 2026-02-17T12:12:12.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:13.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:12:13.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:13.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-17T12:12:13.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:12:13.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:13.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:13.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 == 0 2026-02-17T12:12:13.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:12:13.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:12:13.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:12:13.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:12:13.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:12:13.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:12:13.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:12:13.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=5 2026-02-17T12:12:13.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:12:13.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:13.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:13.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 5 = 5 2026-02-17T12:12:13.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:12:13.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:12:13.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 5 = 1 ']' 2026-02-17T12:12:13.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T12:12:13.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 5 = 6 ']' 2026-02-17T12:12:13.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T12:12:13.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-6 1 1 2026-02-17T12:12:13.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-6 1 1 2026-02-17T12:12:14.056 INFO:tasks.workunit.client.0.trial128.stderr:pool 'psr_pool-6' created 2026-02-17T12:12:14.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:12:15.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T12:12:15.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:12:15.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:12:15.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:12:15.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:12:15.070 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:12:15.071 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:12:15.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:12:15.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:12:15.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:12:15.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:12:15.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:12:15.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:12:15.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:12:15.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:12:15.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:12:15.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T12:12:15.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:12:15.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:15.347 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:12:15.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836491 2026-02-17T12:12:15.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836491 2026-02-17T12:12:15.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491' 2026-02-17T12:12:15.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:15.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836491 2026-02-17T12:12:15.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:15.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:12:15.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836491 2026-02-17T12:12:15.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:15.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836491 2026-02-17T12:12:15.422 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836491 2026-02-17T12:12:15.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-17T12:12:15.423 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:15.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-17T12:12:15.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:16.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:12:16.590 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:16.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-17T12:12:16.741 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:12:16.742 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:16.742 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:16.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 6 == 0 2026-02-17T12:12:16.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:12:16.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:12:16.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:12:16.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:12:16.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:12:16.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:12:16.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:12:17.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=6 2026-02-17T12:12:17.077 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:12:17.078 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:17.078 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:17.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 6 = 6 2026-02-17T12:12:17.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:12:17.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:12:17.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 6 = 1 ']' 2026-02-17T12:12:17.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T12:12:17.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 6 = 6 ']' 2026-02-17T12:12:17.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5868: TEST_scrub_warning: ceph osd pool set psr_pool-6 deep_scrub_interval 1209600 2026-02-17T12:12:17.511 INFO:tasks.workunit.client.0.trial128.stderr:set pool 6 deep_scrub_interval to 1209600 2026-02-17T12:12:17.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T12:12:17.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-7 1 1 2026-02-17T12:12:17.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-7 1 1 2026-02-17T12:12:17.714 INFO:tasks.workunit.client.0.trial128.stderr:pool 'psr_pool-7' created 2026-02-17T12:12:17.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:12:18.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T12:12:18.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:12:18.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:12:18.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:12:18.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:12:18.728 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:12:18.729 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:12:18.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:12:18.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:12:18.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:12:18.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:12:18.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:12:18.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:12:18.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:12:18.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:12:18.848 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:12:19.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T12:12:19.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:12:19.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:19.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:12:19.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836493 2026-02-17T12:12:19.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836493 2026-02-17T12:12:19.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836493' 2026-02-17T12:12:19.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:19.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836493 2026-02-17T12:12:19.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:19.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:12:19.077 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836493 2026-02-17T12:12:19.077 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:19.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836493 2026-02-17T12:12:19.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-17T12:12:19.079 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836493 2026-02-17T12:12:19.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:19.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836493 2026-02-17T12:12:19.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:20.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:12:20.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:20.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836493 2026-02-17T12:12:20.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:21.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:12:21.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:21.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836493 2026-02-17T12:12:21.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:12:21.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:21.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:21.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 7 == 0 2026-02-17T12:12:21.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:12:21.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:12:21.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:12:21.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:12:21.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:12:21.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:12:21.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:12:21.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=7 2026-02-17T12:12:21.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:12:21.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:21.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:22.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 7 = 7 2026-02-17T12:12:22.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:12:22.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:12:22.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 7 = 1 ']' 2026-02-17T12:12:22.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T12:12:22.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 7 = 6 ']' 2026-02-17T12:12:22.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T12:12:22.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-8 1 1 2026-02-17T12:12:22.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-8 1 1 2026-02-17T12:12:22.265 INFO:tasks.workunit.client.0.trial128.stderr:pool 'psr_pool-8' created 2026-02-17T12:12:22.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:12:23.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T12:12:23.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:12:23.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:12:23.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:12:23.278 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:12:23.278 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:12:23.279 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:12:23.279 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:12:23.279 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:12:23.280 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:12:23.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:12:23.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:12:23.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:12:23.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:12:23.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:12:23.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:12:23.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T12:12:23.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:12:23.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:23.549 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:12:23.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836495 2026-02-17T12:12:23.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836495 2026-02-17T12:12:23.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836495' 2026-02-17T12:12:23.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:23.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836495 2026-02-17T12:12:23.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:23.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:12:23.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836495 2026-02-17T12:12:23.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:23.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836495 2026-02-17T12:12:23.630 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836495 2026-02-17T12:12:23.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836495' 2026-02-17T12:12:23.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:23.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836495 2026-02-17T12:12:23.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:24.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:12:24.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:24.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836495 -lt 21474836495 2026-02-17T12:12:24.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:12:24.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:24.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:25.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 8 == 0 2026-02-17T12:12:25.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:12:25.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:12:25.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:12:25.134 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:12:25.134 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:12:25.134 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:12:25.134 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:12:25.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=8 2026-02-17T12:12:25.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:12:25.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:25.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:25.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 8 = 8 2026-02-17T12:12:25.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:12:25.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:12:25.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 8 = 1 ']' 2026-02-17T12:12:25.461 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T12:12:25.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 8 = 6 ']' 2026-02-17T12:12:25.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T12:12:25.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-9 1 1 2026-02-17T12:12:25.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-9 1 1 2026-02-17T12:12:25.650 INFO:tasks.workunit.client.0.trial128.stderr:pool 'psr_pool-9' created 2026-02-17T12:12:25.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:12:26.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T12:12:26.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:12:26.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:12:26.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:12:26.663 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:12:26.663 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:12:26.664 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:12:26.664 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:12:26.664 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:12:26.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:12:26.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:12:26.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:12:26.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:12:26.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:12:26.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:12:26.784 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:12:26.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T12:12:26.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:12:26.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:26.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:12:27.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836496 2026-02-17T12:12:27.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836496 2026-02-17T12:12:27.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836496' 2026-02-17T12:12:27.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:27.012 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836496 2026-02-17T12:12:27.012 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:27.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:12:27.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836496 2026-02-17T12:12:27.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:27.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836496 2026-02-17T12:12:27.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836496' 2026-02-17T12:12:27.017 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836496 2026-02-17T12:12:27.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:27.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836495 -lt 21474836496 2026-02-17T12:12:27.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:28.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:12:28.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:28.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836495 -lt 21474836496 2026-02-17T12:12:28.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:29.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:12:29.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:29.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836497 -lt 21474836496 2026-02-17T12:12:29.497 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:12:29.497 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:29.497 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:29.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 9 == 0 2026-02-17T12:12:29.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:12:29.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:12:29.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:12:29.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:12:29.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:12:29.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:12:29.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:12:29.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=9 2026-02-17T12:12:29.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:12:29.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:29.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:30.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 9 = 9 2026-02-17T12:12:30.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:12:30.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:12:30.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 9 = 1 ']' 2026-02-17T12:12:30.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T12:12:30.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 9 = 6 ']' 2026-02-17T12:12:30.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5858: TEST_scrub_warning: for i in $(seq 1 $(expr $scrubs + $deep_scrubs)) 2026-02-17T12:12:30.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5860: TEST_scrub_warning: create_pool psr_pool-10 1 1 2026-02-17T12:12:30.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create psr_pool-10 1 1 2026-02-17T12:12:30.236 INFO:tasks.workunit.client.0.trial128.stderr:pool 'psr_pool-10' created 2026-02-17T12:12:30.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:12:31.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5861: TEST_scrub_warning: wait_for_clean 2026-02-17T12:12:31.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:12:31.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:12:31.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:12:31.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:12:31.249 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:12:31.249 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:12:31.250 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:12:31.250 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:12:31.250 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:12:31.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:12:31.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:12:31.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:12:31.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:12:31.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:12:31.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:12:31.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T12:12:31.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:12:31.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:31.519 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:12:31.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836498 2026-02-17T12:12:31.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836498 2026-02-17T12:12:31.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498' 2026-02-17T12:12:31.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:31.591 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836498 2026-02-17T12:12:31.591 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:31.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:12:31.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836498 2026-02-17T12:12:31.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:31.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836498 2026-02-17T12:12:31.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836498' 2026-02-17T12:12:31.595 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836498 2026-02-17T12:12:31.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:31.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836497 -lt 21474836498 2026-02-17T12:12:31.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:32.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:12:32.750 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:32.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836498 -lt 21474836498 2026-02-17T12:12:32.909 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:12:32.910 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:32.910 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:33.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 10 == 0 2026-02-17T12:12:33.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:12:33.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:12:33.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:12:33.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:12:33.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:12:33.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:12:33.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:12:33.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=10 2026-02-17T12:12:33.251 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:12:33.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:12:33.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:12:33.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 10 = 10 2026-02-17T12:12:33.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:12:33.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:12:33.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5862: TEST_scrub_warning: '[' 10 = 1 ']' 2026-02-17T12:12:33.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: expr 5 + 1 2026-02-17T12:12:33.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5866: TEST_scrub_warning: '[' 10 = 6 ']' 2026-02-17T12:12:33.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5873: TEST_scrub_warning: local primary=0 2026-02-17T12:12:33.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5875: TEST_scrub_warning: ceph osd set noscrub 2026-02-17T12:12:33.638 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:12:33.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5876: TEST_scrub_warning: ceph osd set nodeep-scrub 2026-02-17T12:12:33.842 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:12:33.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5877: TEST_scrub_warning: ceph config set global osd_scrub_interval_randomize_ratio 0 2026-02-17T12:12:34.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5878: TEST_scrub_warning: ceph config set global osd_deep_scrub_randomize_ratio 0 2026-02-17T12:12:34.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5879: TEST_scrub_warning: ceph config set global osd_scrub_max_interval 604800 2026-02-17T12:12:34.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5880: TEST_scrub_warning: ceph config set global osd_deep_scrub_interval 604800 2026-02-17T12:12:34.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5883: TEST_scrub_warning: seq 1 5 2026-02-17T12:12:34.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5883: TEST_scrub_warning: for i in $(seq 1 $scrubs) 2026-02-17T12:12:34.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5885: TEST_scrub_warning: '[' 1 = 1 ']' 2026-02-17T12:12:34.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5887: TEST_scrub_warning: overdue_seconds=1900800 2026-02-17T12:12:34.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: expr 1900800 + 100 2026-02-17T12:12:34.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: ceph tell 1.0 schedule-scrub 1900900 2026-02-17T12:12:34.586 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:12:34.586 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:12:34.586 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:12:34.586 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-01-26T12:09:14.580792+0000" 2026-02-17T12:12:34.586 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:12:34.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5883: TEST_scrub_warning: for i in $(seq 1 $scrubs) 2026-02-17T12:12:34.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5885: TEST_scrub_warning: '[' 2 = 1 ']' 2026-02-17T12:12:34.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5889: TEST_scrub_warning: overdue_seconds=993600 2026-02-17T12:12:34.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: expr 993600 + 200 2026-02-17T12:12:34.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: ceph tell 2.0 schedule-scrub 993800 2026-02-17T12:12:34.661 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:12:34.661 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:12:34.661 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:12:34.661 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-06T00:07:34.656404+0000" 2026-02-17T12:12:34.661 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:12:34.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5883: TEST_scrub_warning: for i in $(seq 1 $scrubs) 2026-02-17T12:12:34.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5885: TEST_scrub_warning: '[' 3 = 1 ']' 2026-02-17T12:12:34.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5889: TEST_scrub_warning: overdue_seconds=993600 2026-02-17T12:12:34.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: expr 993600 + 300 2026-02-17T12:12:34.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: ceph tell 3.0 schedule-scrub 993900 2026-02-17T12:12:34.735 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:12:34.735 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:12:34.736 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:12:34.736 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-06T00:05:54.730015+0000" 2026-02-17T12:12:34.736 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:12:34.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5883: TEST_scrub_warning: for i in $(seq 1 $scrubs) 2026-02-17T12:12:34.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5885: TEST_scrub_warning: '[' 4 = 1 ']' 2026-02-17T12:12:34.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5889: TEST_scrub_warning: overdue_seconds=993600 2026-02-17T12:12:34.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: expr 993600 + 400 2026-02-17T12:12:34.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: ceph tell 4.0 schedule-scrub 994000 2026-02-17T12:12:34.811 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:12:34.811 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:12:34.811 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:12:34.811 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-06T00:04:14.805595+0000" 2026-02-17T12:12:34.811 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:12:34.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5883: TEST_scrub_warning: for i in $(seq 1 $scrubs) 2026-02-17T12:12:34.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5885: TEST_scrub_warning: '[' 5 = 1 ']' 2026-02-17T12:12:34.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5889: TEST_scrub_warning: overdue_seconds=993600 2026-02-17T12:12:34.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: expr 993600 + 500 2026-02-17T12:12:34.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5891: TEST_scrub_warning: ceph tell 5.0 schedule-scrub 994100 2026-02-17T12:12:34.886 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:12:34.886 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:12:34.887 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:12:34.887 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-06T00:02:34.881207+0000" 2026-02-17T12:12:34.887 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:12:34.894 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5894: TEST_scrub_warning: expr 5 + 1 2026-02-17T12:12:34.896 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5894: TEST_scrub_warning: expr 5 + 5 2026-02-17T12:12:34.898 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5894: TEST_scrub_warning: seq 6 10 2026-02-17T12:12:34.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5894: TEST_scrub_warning: for i in $(seq $(expr $scrubs + 1) $(expr $scrubs + $deep_scrubs)) 2026-02-17T12:12:34.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: expr 5 + 1 2026-02-17T12:12:34.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: '[' 6 = 6 ']' 2026-02-17T12:12:34.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5898: TEST_scrub_warning: overdue_seconds=1900800 2026-02-17T12:12:34.902 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: expr 1900800 + 600 2026-02-17T12:12:34.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: ceph tell 6.0 schedule-deep-scrub 1901400 2026-02-17T12:12:34.971 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:12:34.971 INFO:tasks.workunit.client.0.trial128.stdout: "deep": true, 2026-02-17T12:12:34.971 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:12:34.971 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-01-26T12:00:54.965551+0000" 2026-02-17T12:12:34.971 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:12:34.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5894: TEST_scrub_warning: for i in $(seq $(expr $scrubs + 1) $(expr $scrubs + $deep_scrubs)) 2026-02-17T12:12:34.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: expr 5 + 1 2026-02-17T12:12:34.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: '[' 7 = 6 ']' 2026-02-17T12:12:34.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5900: TEST_scrub_warning: overdue_seconds=993600 2026-02-17T12:12:34.980 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: expr 993600 + 700 2026-02-17T12:12:34.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: ceph tell 7.0 schedule-deep-scrub 994300 2026-02-17T12:12:35.046 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:12:35.046 INFO:tasks.workunit.client.0.trial128.stdout: "deep": true, 2026-02-17T12:12:35.046 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:12:35.047 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-05T23:59:15.040976+0000" 2026-02-17T12:12:35.047 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:12:35.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5894: TEST_scrub_warning: for i in $(seq $(expr $scrubs + 1) $(expr $scrubs + $deep_scrubs)) 2026-02-17T12:12:35.054 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: expr 5 + 1 2026-02-17T12:12:35.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: '[' 8 = 6 ']' 2026-02-17T12:12:35.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5900: TEST_scrub_warning: overdue_seconds=993600 2026-02-17T12:12:35.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: expr 993600 + 800 2026-02-17T12:12:35.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: ceph tell 8.0 schedule-deep-scrub 994400 2026-02-17T12:12:35.122 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:12:35.122 INFO:tasks.workunit.client.0.trial128.stdout: "deep": true, 2026-02-17T12:12:35.122 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:12:35.122 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-05T23:57:35.116437+0000" 2026-02-17T12:12:35.122 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:12:35.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5894: TEST_scrub_warning: for i in $(seq $(expr $scrubs + 1) $(expr $scrubs + $deep_scrubs)) 2026-02-17T12:12:35.129 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: expr 5 + 1 2026-02-17T12:12:35.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: '[' 9 = 6 ']' 2026-02-17T12:12:35.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5900: TEST_scrub_warning: overdue_seconds=993600 2026-02-17T12:12:35.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: expr 993600 + 900 2026-02-17T12:12:35.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: ceph tell 9.0 schedule-deep-scrub 994500 2026-02-17T12:12:35.198 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:12:35.198 INFO:tasks.workunit.client.0.trial128.stdout: "deep": true, 2026-02-17T12:12:35.198 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:12:35.198 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-05T23:55:55.192684+0000" 2026-02-17T12:12:35.198 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:12:35.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5894: TEST_scrub_warning: for i in $(seq $(expr $scrubs + 1) $(expr $scrubs + $deep_scrubs)) 2026-02-17T12:12:35.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: expr 5 + 1 2026-02-17T12:12:35.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5896: TEST_scrub_warning: '[' 10 = 6 ']' 2026-02-17T12:12:35.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5900: TEST_scrub_warning: overdue_seconds=993600 2026-02-17T12:12:35.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: expr 993600 + 1000 2026-02-17T12:12:35.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5902: TEST_scrub_warning: ceph tell 10.0 schedule-deep-scrub 994600 2026-02-17T12:12:35.274 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:12:35.274 INFO:tasks.workunit.client.0.trial128.stdout: "deep": true, 2026-02-17T12:12:35.274 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:12:35.274 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-05T23:54:15.268674+0000" 2026-02-17T12:12:35.275 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:12:35.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5904: TEST_scrub_warning: flush_pg_stats 2026-02-17T12:12:35.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:12:35.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:12:35.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T12:12:35.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:12:35.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:35.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:12:35.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836500 2026-02-17T12:12:35.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836500 2026-02-17T12:12:35.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836500' 2026-02-17T12:12:35.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:35.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836500 2026-02-17T12:12:35.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:35.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:12:35.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836500 2026-02-17T12:12:35.525 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:35.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836500 2026-02-17T12:12:35.527 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836500 2026-02-17T12:12:35.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836500' 2026-02-17T12:12:35.527 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:35.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836500 2026-02-17T12:12:35.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:36.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:12:36.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:36.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836500 -lt 21474836500 2026-02-17T12:12:36.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5906: TEST_scrub_warning: ceph health 2026-02-17T12:12:37.012 INFO:tasks.workunit.client.0.trial128.stdout:HEALTH_WARN noscrub,nodeep-scrub flag(s) set; 5 pgs not deep-scrubbed in time; 10 pgs not scrubbed in time; 10 pool(s) do not have an application enabled; 10 pool(s) have no replicas configured; OSD count 1 < osd_pool_default_size 3 2026-02-17T12:12:37.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5907: TEST_scrub_warning: ceph health detail 2026-02-17T12:12:37.209 INFO:tasks.workunit.client.0.trial128.stdout:HEALTH_WARN noscrub,nodeep-scrub flag(s) set; 5 pgs not deep-scrubbed in time; 10 pgs not scrubbed in time; 10 pool(s) do not have an application enabled; 10 pool(s) have no replicas configured; OSD count 1 < osd_pool_default_size 3 2026-02-17T12:12:37.209 INFO:tasks.workunit.client.0.trial128.stdout:[WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2026-02-17T12:12:37.209 INFO:tasks.workunit.client.0.trial128.stdout:[WRN] PG_NOT_DEEP_SCRUBBED: 5 pgs not deep-scrubbed in time 2026-02-17T12:12:37.209 INFO:tasks.workunit.client.0.trial128.stdout: pg 10.0 not deep-scrubbed since 2026-02-05T23:54:15.268674+0000 2026-02-17T12:12:37.210 INFO:tasks.workunit.client.0.trial128.stdout: pg 9.0 not deep-scrubbed since 2026-02-05T23:55:55.192684+0000 2026-02-17T12:12:37.210 INFO:tasks.workunit.client.0.trial128.stdout: pg 8.0 not deep-scrubbed since 2026-02-05T23:57:35.116437+0000 2026-02-17T12:12:37.210 INFO:tasks.workunit.client.0.trial128.stdout: pg 7.0 not deep-scrubbed since 2026-02-05T23:59:15.040976+0000 2026-02-17T12:12:37.210 INFO:tasks.workunit.client.0.trial128.stdout: pg 6.0 not deep-scrubbed since 2026-01-26T12:00:54.965551+0000 2026-02-17T12:12:37.210 INFO:tasks.workunit.client.0.trial128.stdout:[WRN] PG_NOT_SCRUBBED: 10 pgs not scrubbed in time 2026-02-17T12:12:37.210 INFO:tasks.workunit.client.0.trial128.stdout: pg 10.0 not scrubbed since 2026-02-05T23:54:15.268674+0000 2026-02-17T12:12:37.210 INFO:tasks.workunit.client.0.trial128.stdout: pg 9.0 not scrubbed since 2026-02-05T23:55:55.192684+0000 2026-02-17T12:12:37.210 INFO:tasks.workunit.client.0.trial128.stdout: pg 8.0 not scrubbed since 2026-02-05T23:57:35.116437+0000 2026-02-17T12:12:37.210 INFO:tasks.workunit.client.0.trial128.stdout: pg 7.0 not scrubbed since 2026-02-05T23:59:15.040976+0000 2026-02-17T12:12:37.210 INFO:tasks.workunit.client.0.trial128.stdout: pg 6.0 not scrubbed since 2026-01-26T12:00:54.965551+0000 2026-02-17T12:12:37.210 INFO:tasks.workunit.client.0.trial128.stdout: pg 5.0 not scrubbed since 2026-02-06T00:02:34.881207+0000 2026-02-17T12:12:37.211 INFO:tasks.workunit.client.0.trial128.stdout: pg 4.0 not scrubbed since 2026-02-06T00:04:14.805595+0000 2026-02-17T12:12:37.211 INFO:tasks.workunit.client.0.trial128.stdout: pg 3.0 not scrubbed since 2026-02-06T00:05:54.730015+0000 2026-02-17T12:12:37.211 INFO:tasks.workunit.client.0.trial128.stdout: pg 2.0 not scrubbed since 2026-02-06T00:07:34.656404+0000 2026-02-17T12:12:37.211 INFO:tasks.workunit.client.0.trial128.stdout: pg 1.0 not scrubbed since 2026-01-26T12:09:14.580792+0000 2026-02-17T12:12:37.211 INFO:tasks.workunit.client.0.trial128.stdout:[WRN] POOL_APP_NOT_ENABLED: 10 pool(s) do not have an application enabled 2026-02-17T12:12:37.211 INFO:tasks.workunit.client.0.trial128.stdout: application not enabled on pool 'psr_pool-1' 2026-02-17T12:12:37.211 INFO:tasks.workunit.client.0.trial128.stdout: application not enabled on pool 'psr_pool-2' 2026-02-17T12:12:37.211 INFO:tasks.workunit.client.0.trial128.stdout: application not enabled on pool 'psr_pool-3' 2026-02-17T12:12:37.211 INFO:tasks.workunit.client.0.trial128.stdout: application not enabled on pool 'psr_pool-4' 2026-02-17T12:12:37.211 INFO:tasks.workunit.client.0.trial128.stdout: application not enabled on pool 'psr_pool-5' 2026-02-17T12:12:37.211 INFO:tasks.workunit.client.0.trial128.stdout: application not enabled on pool 'psr_pool-6' 2026-02-17T12:12:37.211 INFO:tasks.workunit.client.0.trial128.stdout: application not enabled on pool 'psr_pool-7' 2026-02-17T12:12:37.211 INFO:tasks.workunit.client.0.trial128.stdout: application not enabled on pool 'psr_pool-8' 2026-02-17T12:12:37.212 INFO:tasks.workunit.client.0.trial128.stdout: application not enabled on pool 'psr_pool-9' 2026-02-17T12:12:37.212 INFO:tasks.workunit.client.0.trial128.stdout: application not enabled on pool 'psr_pool-10' 2026-02-17T12:12:37.212 INFO:tasks.workunit.client.0.trial128.stdout: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2026-02-17T12:12:37.212 INFO:tasks.workunit.client.0.trial128.stdout:[WRN] POOL_NO_REDUNDANCY: 10 pool(s) have no replicas configured 2026-02-17T12:12:37.212 INFO:tasks.workunit.client.0.trial128.stdout: pool 'psr_pool-1' has no replicas configured 2026-02-17T12:12:37.212 INFO:tasks.workunit.client.0.trial128.stdout: pool 'psr_pool-2' has no replicas configured 2026-02-17T12:12:37.212 INFO:tasks.workunit.client.0.trial128.stdout: pool 'psr_pool-3' has no replicas configured 2026-02-17T12:12:37.212 INFO:tasks.workunit.client.0.trial128.stdout: pool 'psr_pool-4' has no replicas configured 2026-02-17T12:12:37.212 INFO:tasks.workunit.client.0.trial128.stdout: pool 'psr_pool-5' has no replicas configured 2026-02-17T12:12:37.212 INFO:tasks.workunit.client.0.trial128.stdout: pool 'psr_pool-6' has no replicas configured 2026-02-17T12:12:37.212 INFO:tasks.workunit.client.0.trial128.stdout: pool 'psr_pool-7' has no replicas configured 2026-02-17T12:12:37.212 INFO:tasks.workunit.client.0.trial128.stdout: pool 'psr_pool-8' has no replicas configured 2026-02-17T12:12:37.213 INFO:tasks.workunit.client.0.trial128.stdout: pool 'psr_pool-9' has no replicas configured 2026-02-17T12:12:37.213 INFO:tasks.workunit.client.0.trial128.stdout: pool 'psr_pool-10' has no replicas configured 2026-02-17T12:12:37.213 INFO:tasks.workunit.client.0.trial128.stdout:[WRN] TOO_FEW_OSDS: OSD count 1 < osd_pool_default_size 3 2026-02-17T12:12:37.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5908: TEST_scrub_warning: ceph health 2026-02-17T12:12:37.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5908: TEST_scrub_warning: grep -q ' pgs not deep-scrubbed in time' 2026-02-17T12:12:37.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5909: TEST_scrub_warning: ceph health 2026-02-17T12:12:37.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5909: TEST_scrub_warning: grep -q ' pgs not scrubbed in time' 2026-02-17T12:12:37.585 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5914: TEST_scrub_warning: ceph health detail 2026-02-17T12:12:37.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5914: TEST_scrub_warning: grep 'not scrubbed since' 2026-02-17T12:12:37.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5914: TEST_scrub_warning: wc -l 2026-02-17T12:12:37.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5914: TEST_scrub_warning: COUNT=10 2026-02-17T12:12:37.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5916: TEST_scrub_warning: expr 5+5 2026-02-17T12:12:37.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5916: TEST_scrub_warning: (( 10 != 5 && 10 != 5+5 )) 2026-02-17T12:12:37.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5920: TEST_scrub_warning: ceph health detail 2026-02-17T12:12:37.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5920: TEST_scrub_warning: grep 'not deep-scrubbed since' 2026-02-17T12:12:37.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5920: TEST_scrub_warning: wc -l 2026-02-17T12:12:37.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5920: TEST_scrub_warning: COUNT=5 2026-02-17T12:12:37.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:5921: TEST_scrub_warning: '[' 5 '!=' 5 ']' 2026-02-17T12:12:37.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T12:12:37.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:12:37.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:12:37.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:12:37.962 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:12:37.962 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:12:37.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:12:37.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:12:37.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:12:38.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:12:38.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:12:38.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:12:38.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:12:38.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:12:38.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:12:38.074 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:12:38.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:12:38.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:12:38.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:12:38.077 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:12:38.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:12:38.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:12:38.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:12:38.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:12:38.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:38.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:12:38.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:12:38.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:12:38.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:12:38.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T12:12:38.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T12:12:38.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T12:12:38.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T12:12:38.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:12:38.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:12:38.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:12:38.092 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:12:38.092 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:12:38.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:12:38.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:12:38.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:12:38.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:12:38.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:12:38.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:12:38.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:12:38.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:12:38.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:12:38.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:12:38.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:12:38.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:12:38.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:12:38.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:12:38.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:12:38.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:12:38.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:12:38.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:12:38.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:38.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:12:38.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:12:38.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:12:38.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:12:38.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T12:12:38.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:12:38.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:38.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:12:38.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T12:12:38.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:12:38.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:12:38.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:12:38.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T12:12:38.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_skip_non_repair_during_recovery td/osd-scrub-repair 2026-02-17T12:12:38.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:136: TEST_skip_non_repair_during_recovery: local dir=td/osd-scrub-repair 2026-02-17T12:12:38.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:137: TEST_skip_non_repair_during_recovery: local poolname=rbd 2026-02-17T12:12:38.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:139: TEST_skip_non_repair_during_recovery: run_mon td/osd-scrub-repair a --osd_pool_default_size=2 2026-02-17T12:12:38.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T12:12:38.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:12:38.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:12:38.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:12:38.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T12:12:38.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair --osd_pool_default_size=2 2026-02-17T12:12:38.132 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:12:38.132 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:12:38.132 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:12:38.133 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:12:38.133 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:38.133 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:12:38.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:12:38.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T12:12:38.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:12:38.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:12:38.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:12:38.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:12:38.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:12:38.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:12:38.172 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:12:38.172 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:12:38.172 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:12:38.173 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:12:38.173 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:38.174 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:12:38.174 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:12:38.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:12:38.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T12:12:38.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:12:38.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:12:38.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:12:38.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:12:38.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:12:38.233 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:12:38.233 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:12:38.233 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:12:38.233 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:12:38.234 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:38.234 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:12:38.234 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:12:38.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:12:38.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T12:12:38.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:140: TEST_skip_non_repair_during_recovery: run_mgr td/osd-scrub-repair x 2026-02-17T12:12:38.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T12:12:38.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:12:38.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:12:38.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:12:38.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T12:12:38.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:12:38.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:12:38.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:12:38.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:12:38.387 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:12:38.387 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:38.387 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:12:38.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:12:38.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:12:38.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:12:38.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:141: TEST_skip_non_repair_during_recovery: run_osd td/osd-scrub-repair 0 --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true 2026-02-17T12:12:38.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:12:38.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:12:38.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:12:38.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:12:38.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:12:38.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:12:38.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:12:38.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:12:38.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:12:38.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:12:38.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:12:38.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:12:38.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:12:38.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:12:38.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:12:38.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:12:38.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:12:38.412 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:12:38.412 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:38.412 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:12:38.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:12:38.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:12:38.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:12:38.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:12:38.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:12:38.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:12:38.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:12:38.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:12:38.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:12:38.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:12:38.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:12:38.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:12:38.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true' 2026-02-17T12:12:38.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:12:38.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:12:38.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=581f928f-7fc5-4cdf-90fe-9a5e5998cfa8 2026-02-17T12:12:38.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 581f928f-7fc5-4cdf-90fe-9a5e5998cfa8' 2026-02-17T12:12:38.415 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 581f928f-7fc5-4cdf-90fe-9a5e5998cfa8 2026-02-17T12:12:38.415 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:12:38.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA2W5RpOo/tGBAAaLRgwxpb9g1iqVKCeA51Xg== 2026-02-17T12:12:38.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA2W5RpOo/tGBAAaLRgwxpb9g1iqVKCeA51Xg=="}' 2026-02-17T12:12:38.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 581f928f-7fc5-4cdf-90fe-9a5e5998cfa8 -i td/osd-scrub-repair/0/new.json 2026-02-17T12:12:38.515 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:12:38.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T12:12:38.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true --mkfs --key AQA2W5RpOo/tGBAAaLRgwxpb9g1iqVKCeA51Xg== --osd-uuid 581f928f-7fc5-4cdf-90fe-9a5e5998cfa8 2026-02-17T12:12:38.540 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:38.531+0000 7fb8afd868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:38.542 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:38.531+0000 7fb8afd868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:38.543 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:38.531+0000 7fb8afd868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:38.543 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:38.535+0000 7fb8afd868c0 -1 bdev(0x558834da3c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:12:38.543 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:38.535+0000 7fb8afd868c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T12:12:41.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T12:12:41.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:12:41.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:12:41.186 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:12:41.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:12:41.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:12:41.383 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:12:41.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true 2026-02-17T12:12:41.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:12:41.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:12:41.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:12:41.408 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:41.399+0000 7f77e2fbd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:41.410 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:41.399+0000 7f77e2fbd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:41.411 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:41.399+0000 7f77e2fbd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:41.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:12:41.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:12:41.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:12:41.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:12:41.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:12:41.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:41.535 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:12:41.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:12:41.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:41.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:12:41.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:42.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:42.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:42.692 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:12:42.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:12:42.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:42.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:12:42.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:42.889 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:42.879+0000 7f77e2fbd8c0 -1 Falling back to public interface 2026-02-17T12:12:43.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:43.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:43.848 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:12:43.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:12:43.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:43.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:12:44.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:45.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:45.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:45.009 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:12:45.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:12:45.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:45.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:12:45.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:45.904 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:45.895+0000 7f77e2fbd8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:12:46.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:46.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:46.163 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:12:46.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:12:46.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:46.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:12:46.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:47.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:47.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:47.323 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:12:47.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:12:47.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:47.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:12:47.479 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3427777528,v1:127.0.0.1:6803/3427777528] [v2:127.0.0.1:6804/3427777528,v1:127.0.0.1:6805/3427777528] exists,up 581f928f-7fc5-4cdf-90fe-9a5e5998cfa8 2026-02-17T12:12:47.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:12:47.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:12:47.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:12:47.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:144: TEST_skip_non_repair_during_recovery: run_osd td/osd-scrub-repair 1 --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true 2026-02-17T12:12:47.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:12:47.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:12:47.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:12:47.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:12:47.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:12:47.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:12:47.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:12:47.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:12:47.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:12:47.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:12:47.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:12:47.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:12:47.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:12:47.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:12:47.483 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:12:47.483 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:12:47.483 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:12:47.483 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:12:47.483 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:12:47.483 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:12:47.483 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:12:47.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:12:47.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:12:47.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:12:47.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:12:47.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:12:47.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:12:47.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:12:47.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:12:47.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:12:47.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:12:47.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:12:47.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true' 2026-02-17T12:12:47.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:12:47.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:12:47.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=9c777055-542f-4cd7-b86a-8aa6c570e7ae 2026-02-17T12:12:47.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 9c777055-542f-4cd7-b86a-8aa6c570e7ae' 2026-02-17T12:12:47.487 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 9c777055-542f-4cd7-b86a-8aa6c570e7ae 2026-02-17T12:12:47.487 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:12:47.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA/W5RpST5JHRAAjHXmJZA6bzNtvunTY5+K5Q== 2026-02-17T12:12:47.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA/W5RpST5JHRAAjHXmJZA6bzNtvunTY5+K5Q=="}' 2026-02-17T12:12:47.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 9c777055-542f-4cd7-b86a-8aa6c570e7ae -i td/osd-scrub-repair/1/new.json 2026-02-17T12:12:47.660 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:12:47.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T12:12:47.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true --mkfs --key AQA/W5RpST5JHRAAjHXmJZA6bzNtvunTY5+K5Q== --osd-uuid 9c777055-542f-4cd7-b86a-8aa6c570e7ae 2026-02-17T12:12:47.688 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:47.679+0000 7f520b7608c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:47.690 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:47.679+0000 7f520b7608c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:47.691 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:47.679+0000 7f520b7608c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:47.691 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:47.683+0000 7f520b7608c0 -1 bdev(0x5619effd1c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:12:47.691 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:47.683+0000 7f520b7608c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T12:12:50.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T12:12:50.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:12:50.203 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:12:50.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:12:50.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:12:50.385 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:12:50.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:12:50.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_during_recovery=false --osd_repair_during_recovery=true --osd_debug_pretend_recovery_active=true 2026-02-17T12:12:50.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:12:50.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:12:50.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:12:50.401 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:50.391+0000 7fb103b5c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:50.402 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:50.391+0000 7fb103b5c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:50.403 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:50.391+0000 7fb103b5c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:12:50.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:12:50.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:12:50.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:12:50.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:12:50.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:12:50.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:50.543 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:12:50.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:12:50.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:50.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:12:50.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:51.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:51.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:51.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:12:51.696 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:12:51.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:51.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:12:51.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:51.925 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:51.915+0000 7fb103b5c8c0 -1 Falling back to public interface 2026-02-17T12:12:52.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:52.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:52.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:12:52.855 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:12:52.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:52.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:12:53.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:54.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:54.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:54.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:12:54.011 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:12:54.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:54.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:12:54.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:54.895 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:12:54.883+0000 7fb103b5c8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:12:55.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:55.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:55.164 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:12:55.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:12:55.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:55.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:12:55.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:12:56.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:12:56.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:12:56.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:12:56.322 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:12:56.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:12:56.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:12:56.475 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2471814356,v1:127.0.0.1:6811/2471814356] [v2:127.0.0.1:6812/2471814356,v1:127.0.0.1:6813/2471814356] exists,up 9c777055-542f-4cd7-b86a-8aa6c570e7ae 2026-02-17T12:12:56.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:12:56.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:12:56.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:12:56.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:147: TEST_skip_non_repair_during_recovery: create_rbd_pool 2026-02-17T12:12:56.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:535: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-17T12:12:56.621 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' does not exist 2026-02-17T12:12:56.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:536: create_rbd_pool: create_pool rbd 4 2026-02-17T12:12:56.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create rbd 4 2026-02-17T12:12:56.823 INFO:tasks.workunit.client.0.trial128.stderr:pool 'rbd' created 2026-02-17T12:12:56.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:12:57.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: create_rbd_pool: rbd pool init rbd 2026-02-17T12:12:58.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:148: TEST_skip_non_repair_during_recovery: wait_for_clean 2026-02-17T12:12:58.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:12:58.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:12:58.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:12:58.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:12:58.116 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:12:58.116 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:12:58.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:12:58.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:12:58.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:12:58.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:12:58.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:12:58.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:12:58.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:12:58.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:12:58.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:12:58.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:12:58.388 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:12:58.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:12:58.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:58.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:12:58.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T12:12:58.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T12:12:58.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T12:12:58.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:12:58.461 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:12:58.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672962 2026-02-17T12:12:58.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672962 2026-02-17T12:12:58.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-42949672962' 2026-02-17T12:12:58.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:58.536 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T12:12:58.536 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:58.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:12:58.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T12:12:58.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:58.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T12:12:58.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T12:12:58.541 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836484 2026-02-17T12:12:58.541 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:58.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T12:12:58.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:12:59.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:12:59.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:12:59.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T12:12:59.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:12:59.851 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672962 2026-02-17T12:12:59.851 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:12:59.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:12:59.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672962 2026-02-17T12:12:59.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:12:59.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672962 2026-02-17T12:12:59.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672962' 2026-02-17T12:12:59.856 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672962 2026-02-17T12:12:59.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:13:00.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672962 -lt 42949672962 2026-02-17T12:13:00.021 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:13:00.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:13:00.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:13:00.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T12:13:00.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:13:00.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:13:00.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:13:00.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:13:00.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:13:00.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:13:00.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:13:00.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T12:13:00.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:13:00.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:13:00.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:13:00.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T12:13:00.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:13:00.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:13:00.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:150: TEST_skip_non_repair_during_recovery: add_something td/osd-scrub-repair rbd 2026-02-17T12:13:00.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T12:13:00.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=rbd 2026-02-17T12:13:00.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T12:13:00.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T12:13:00.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T12:13:00.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T12:13:00.750 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:13:00.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T12:13:00.954 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:13:00.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T12:13:00.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T12:13:00.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool rbd put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T12:13:00.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:151: TEST_skip_non_repair_during_recovery: get_not_primary rbd SOMETHING 2026-02-17T12:13:00.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=rbd 2026-02-17T12:13:00.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=SOMETHING 2026-02-17T12:13:00.985 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary rbd SOMETHING 2026-02-17T12:13:00.985 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=rbd 2026-02-17T12:13:00.985 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T12:13:00.986 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map rbd SOMETHING 2026-02-17T12:13:00.986 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:13:01.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T12:13:01.134 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map rbd SOMETHING 2026-02-17T12:13:01.134 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T12:13:01.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:151: TEST_skip_non_repair_during_recovery: scrub_and_not_schedule td/osd-scrub-repair rbd 0 2026-02-17T12:13:01.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:155: scrub_and_not_schedule: local dir=td/osd-scrub-repair 2026-02-17T12:13:01.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:156: scrub_and_not_schedule: local poolname=rbd 2026-02-17T12:13:01.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:157: scrub_and_not_schedule: local osd=0 2026-02-17T12:13:01.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:162: scrub_and_not_schedule: get_pg rbd SOMETHING 2026-02-17T12:13:01.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=rbd 2026-02-17T12:13:01.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T12:13:01.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map rbd SOMETHING 2026-02-17T12:13:01.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T12:13:01.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:162: scrub_and_not_schedule: local pg=1.3 2026-02-17T12:13:01.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:163: scrub_and_not_schedule: get_last_scrub_stamp 1.3 2026-02-17T12:13:01.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T12:13:01.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:13:01.448 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:13:01.448 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T12:13:01.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:163: scrub_and_not_schedule: local last_scrub=2026-02-17T12:12:56.814777+0000 2026-02-17T12:13:01.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:164: scrub_and_not_schedule: ceph tell 1.3 schedule-scrub 2026-02-17T12:13:01.667 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:13:01.667 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:13:01.667 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:13:01.667 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-10T12:11:21.662155+0000" 2026-02-17T12:13:01.668 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:13:01.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: scrub_and_not_schedule: (( i=0 )) 2026-02-17T12:13:01.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: scrub_and_not_schedule: (( i < 3 )) 2026-02-17T12:13:01.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:170: scrub_and_not_schedule: get_last_scrub_stamp 1.3 2026-02-17T12:13:01.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T12:13:01.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:13:01.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:13:01.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T12:13:01.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:170: scrub_and_not_schedule: test 2026-02-17T12:12:56.814777+0000 '>' 2026-02-17T12:12:56.814777+0000 2026-02-17T12:13:01.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:173: scrub_and_not_schedule: sleep 1 2026-02-17T12:13:02.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: scrub_and_not_schedule: (( i++ )) 2026-02-17T12:13:02.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: scrub_and_not_schedule: (( i < 3 )) 2026-02-17T12:13:02.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:170: scrub_and_not_schedule: get_last_scrub_stamp 1.3 2026-02-17T12:13:02.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T12:13:02.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:13:02.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T12:13:02.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:13:02.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:170: scrub_and_not_schedule: test 2026-02-17T12:12:56.814777+0000 '>' 2026-02-17T12:12:56.814777+0000 2026-02-17T12:13:02.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:173: scrub_and_not_schedule: sleep 1 2026-02-17T12:13:03.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: scrub_and_not_schedule: (( i++ )) 2026-02-17T12:13:03.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: scrub_and_not_schedule: (( i < 3 )) 2026-02-17T12:13:03.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:170: scrub_and_not_schedule: get_last_scrub_stamp 1.3 2026-02-17T12:13:03.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.3 2026-02-17T12:13:03.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:13:04.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:13:04.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.3") | .last_scrub_stamp' 2026-02-17T12:13:04.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:170: scrub_and_not_schedule: test 2026-02-17T12:12:56.814777+0000 '>' 2026-02-17T12:12:56.814777+0000 2026-02-17T12:13:04.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:173: scrub_and_not_schedule: sleep 1 2026-02-17T12:13:05.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: scrub_and_not_schedule: (( i++ )) 2026-02-17T12:13:05.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:169: scrub_and_not_schedule: (( i < 3 )) 2026-02-17T12:13:05.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:179: scrub_and_not_schedule: objectstore_tool td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T12:13:05.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:13:05.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:13:05.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T12:13:05.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:13:05.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T12:13:05.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:13:05.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:13:05.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T12:13:05.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:13:05.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T12:13:05.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:13:05.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:13:05.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:13:05.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:13:05.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:13:06.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:13:06.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING list-attrs 2026-02-17T12:13:06.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:13:06.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:13:06.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:13:06.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:13:06.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:13:06.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING list-attrs 2026-02-17T12:13:06.785 INFO:tasks.workunit.client.0.trial128.stdout:_ 2026-02-17T12:13:06.786 INFO:tasks.workunit.client.0.trial128.stdout:snapset 2026-02-17T12:13:07.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T12:13:07.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:13:07.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:13:07.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:13:07.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:13:07.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:13:07.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:13:07.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:13:07.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:13:07.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:13:07.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:13:07.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:13:07.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:13:07.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:13:07.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:13:07.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:13:07.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:07.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:07.324 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:07.324 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:07.324 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:13:07.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:07.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:07.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:13:07.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:13:07.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:13:07.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:13:07.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:13:07.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:13:07.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:13:07.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:13:07.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:13:07.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:13:07.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:13:07.328 INFO:tasks.workunit.client.0.trial128.stderr:start osd.0 2026-02-17T12:13:07.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:13:07.329 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T12:13:07.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:13:07.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:13:07.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:13:07.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:13:07.343 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:07.331+0000 7f412046e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:07.344 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:07.335+0000 7f412046e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:07.345 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:07.335+0000 7f412046e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:07.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:13:07.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:13:07.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:13:07.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:13:07.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:13:07.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:07.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:13:07.484 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:13:07.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:07.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:13:07.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:08.633 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:08.623+0000 7f412046e8c0 -1 Falling back to public interface 2026-02-17T12:13:08.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:08.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:08.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:13:08.636 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:13:08.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:08.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:13:08.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:09.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:09.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:09.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:13:09.797 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:13:09.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:09.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:13:09.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:10.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:10.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:10.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:13:10.949 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:13:10.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:10.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:13:11.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:11.677 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:11.667+0000 7f412046e8c0 -1 osd.0 19 log_to_monitors true 2026-02-17T12:13:12.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:12.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:12.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:13:12.102 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:13:12.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:12.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:13:12.251 INFO:tasks.workunit.client.0.trial128.stderr:osd.0 up in weight 1 up_from 23 up_thru 23 down_at 20 last_clean_interval [5,19) [v2:127.0.0.1:6802/3837059194,v1:127.0.0.1:6803/3837059194] [v2:127.0.0.1:6804/3837059194,v1:127.0.0.1:6805/3837059194] exists,up 581f928f-7fc5-4cdf-90fe-9a5e5998cfa8 2026-02-17T12:13:12.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:13:12.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:13:12.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:13:12.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:13:12.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:13:12.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:13:12.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:13:12.253 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:13:12.253 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:13:12.253 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:13:12.253 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:13:12.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:13:12.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:13:12.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:13:12.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:13:12.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:13:12.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:13:12.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:13:12.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:13:12.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:13:12.521 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:13:12.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:13:12.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:12.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:13:12.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247810 2026-02-17T12:13:12.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247810 2026-02-17T12:13:12.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810' 2026-02-17T12:13:12.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:12.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:13:12.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672966 2026-02-17T12:13:12.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672966 2026-02-17T12:13:12.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-98784247810 1-42949672966' 2026-02-17T12:13:12.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:12.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-98784247810 2026-02-17T12:13:12.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:12.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:13:12.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-98784247810 2026-02-17T12:13:12.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:12.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247810 2026-02-17T12:13:12.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 98784247810' 2026-02-17T12:13:12.670 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 98784247810 2026-02-17T12:13:12.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:13:12.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247810 -lt 98784247810 2026-02-17T12:13:12.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:12.836 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672966 2026-02-17T12:13:12.836 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:12.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:13:12.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672966 2026-02-17T12:13:12.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:12.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672966 2026-02-17T12:13:12.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672966' 2026-02-17T12:13:12.841 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 42949672966 2026-02-17T12:13:12.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:13:12.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672966 -lt 42949672966 2026-02-17T12:13:12.991 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:13:12.992 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:13:12.992 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:13:13.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 == 0 2026-02-17T12:13:13.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:13:13.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:13:13.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:13:13.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:13:13.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:13:13.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:13:13.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:13:13.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=4 2026-02-17T12:13:13.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:13:13.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:13:13.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:13:13.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 4 = 4 2026-02-17T12:13:13.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:13:13.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:13:13.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:180: scrub_and_not_schedule: rados --pool rbd get SOMETHING td/osd-scrub-repair/COPY 2026-02-17T12:13:13.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:181: scrub_and_not_schedule: diff td/osd-scrub-repair/ORIGINAL td/osd-scrub-repair/COPY 2026-02-17T12:13:13.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T12:13:13.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:13:13.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:13:13.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:13:13.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:13:13.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:13:13.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:13:13.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:13:13.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:13:13.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:13:13.641 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:13:13.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:13:13.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:13:13.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:13:13.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:13:13.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:13:13.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:13:13.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:13:13.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:13:13.647 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:13:13.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:13:13.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:13:13.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:13:13.658 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:13:13.658 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:13.658 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:13:13.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:13:13.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:13:13.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:13:13.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T12:13:13.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T12:13:13.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T12:13:13.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T12:13:13.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:13:13.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:13:13.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:13:13.661 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:13:13.661 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:13:13.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:13:13.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:13:13.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:13:13.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:13:13.664 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:13:13.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:13:13.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:13:13.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:13:13.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:13:13.667 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:13:13.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:13:13.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:13:13.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:13:13.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:13:13.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:13:13.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:13:13.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:13:13.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:13:13.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:13.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:13:13.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:13:13.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:13:13.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:13:13.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T12:13:13.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:13:13.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:13.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:13:13.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T12:13:13.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:13:13.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:13:13.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:13:13.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T12:13:13.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_unfound_erasure_coded_appends td/osd-scrub-repair 2026-02-17T12:13:13.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:903: TEST_unfound_erasure_coded_appends: unfound_erasure_coded td/osd-scrub-repair false 2026-02-17T12:13:13.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:852: unfound_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T12:13:13.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:853: unfound_erasure_coded: local allow_overwrites=false 2026-02-17T12:13:13.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:854: unfound_erasure_coded: local poolname=ecpool 2026-02-17T12:13:13.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:855: unfound_erasure_coded: local payload=ABCDEF 2026-02-17T12:13:13.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:857: unfound_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-17T12:13:13.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T12:13:13.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:13:13.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:13:13.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:13:13.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T12:13:13.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T12:13:13.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:13:13.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:13.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:13.703 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:13.703 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:13.703 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:13:13.703 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:13.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:13:13.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:13:13.732 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:13:13.732 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:13:13.732 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:13:13.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:13:13.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:13:13.734 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:13:13.734 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:13:13.734 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:13:13.734 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:13:13.734 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:13.734 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:13:13.735 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:13:13.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:13:13.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T12:13:13.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:13:13.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:13:13.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:13:13.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:13:13.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:13:13.810 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:13:13.810 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:13:13.810 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:13:13.810 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:13:13.810 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:13.811 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:13:13.811 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:13:13.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:13:13.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T12:13:13.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:858: unfound_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-17T12:13:13.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T12:13:13.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:13:13.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:13:13.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:13:13.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T12:13:13.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:13:13.964 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:13:13.964 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:13.964 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:13.964 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:13.964 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:13.964 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:13:13.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:13.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:13:13.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:13:13.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: seq 0 3 2026-02-17T12:13:13.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-17T12:13:13.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:860: unfound_erasure_coded: run_osd td/osd-scrub-repair 0 2026-02-17T12:13:13.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:13:13.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:13:13.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:13:13.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:13:13.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:13:13.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:13:13.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:13:13.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:13:13.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:13:13.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:13:13.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:13:13.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:13:13.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:13:13.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:13:13.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:13:13.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:13.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:13.989 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:13.989 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:13.989 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:13:13.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:13.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:13.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:13:13.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:13:13.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:13:13.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:13:13.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:13:13.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:13:13.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:13:13.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:13:13.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:13:13.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:13:13.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:13:13.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:13:13.992 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:13:13.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=df05ea12-ead7-4d46-916c-cd2203cc573d 2026-02-17T12:13:13.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 df05ea12-ead7-4d46-916c-cd2203cc573d' 2026-02-17T12:13:13.993 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 df05ea12-ead7-4d46-916c-cd2203cc573d 2026-02-17T12:13:13.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:13:14.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBZW5RpDt9ROxAAoeRMkW8qjNGNL4LohUcsIg== 2026-02-17T12:13:14.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBZW5RpDt9ROxAAoeRMkW8qjNGNL4LohUcsIg=="}' 2026-02-17T12:13:14.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new df05ea12-ead7-4d46-916c-cd2203cc573d -i td/osd-scrub-repair/0/new.json 2026-02-17T12:13:14.094 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:13:14.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T12:13:14.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBZW5RpDt9ROxAAoeRMkW8qjNGNL4LohUcsIg== --osd-uuid df05ea12-ead7-4d46-916c-cd2203cc573d 2026-02-17T12:13:14.123 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:14.115+0000 7f38cf0868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:14.125 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:14.115+0000 7f38cf0868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:14.126 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:14.115+0000 7f38cf0868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:14.126 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:14.115+0000 7f38cf0868c0 -1 bdev(0x5579d7ff7c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:13:14.126 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:14.115+0000 7f38cf0868c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T12:13:16.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T12:13:16.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:13:16.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:13:16.510 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:13:16.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:13:16.700 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:13:16.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:13:16.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:13:16.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:13:16.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:13:16.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:13:16.723 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:16.710+0000 7fba4b93a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:16.725 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:16.714+0000 7fba4b93a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:16.725 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:16.714+0000 7fba4b93a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:16.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:13:16.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:13:16.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:13:16.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:13:16.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:13:16.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:16.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:13:16.854 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:13:16.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:16.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:13:17.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:18.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:18.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:18.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:13:18.010 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:13:18.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:18.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:13:18.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:18.221 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:18.210+0000 7fba4b93a8c0 -1 Falling back to public interface 2026-02-17T12:13:19.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:19.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:19.167 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:13:19.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:13:19.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:19.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:13:19.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:20.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:20.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:20.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:13:20.323 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:13:20.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:20.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:13:20.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:21.166 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:21.154+0000 7fba4b93a8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:13:21.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:21.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:21.479 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:13:21.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:13:21.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:21.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:13:21.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:22.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:22.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:22.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:13:22.636 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:13:22.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:22.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:13:22.787 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2075589611,v1:127.0.0.1:6803/2075589611] [v2:127.0.0.1:6804/2075589611,v1:127.0.0.1:6805/2075589611] exists,up df05ea12-ead7-4d46-916c-cd2203cc573d 2026-02-17T12:13:22.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:13:22.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:13:22.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:13:22.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-17T12:13:22.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:860: unfound_erasure_coded: run_osd td/osd-scrub-repair 1 2026-02-17T12:13:22.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:13:22.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:13:22.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:13:22.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:13:22.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:13:22.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:13:22.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:13:22.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:13:22.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:13:22.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:13:22.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:13:22.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:13:22.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:13:22.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:13:22.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:13:22.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:22.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:22.799 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:22.799 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:22.800 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:13:22.800 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:22.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:22.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:13:22.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:13:22.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:13:22.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:13:22.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:13:22.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:13:22.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:13:22.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:13:22.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:13:22.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:13:22.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:13:22.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:13:22.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:13:22.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=0d363fee-fa04-4413-8561-2e808f5c40d2 2026-02-17T12:13:22.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 0d363fee-fa04-4413-8561-2e808f5c40d2' 2026-02-17T12:13:22.803 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 0d363fee-fa04-4413-8561-2e808f5c40d2 2026-02-17T12:13:22.804 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:13:22.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBiW5RpOPycLxAA6LjABDjt5XOWnB1MtRtq2A== 2026-02-17T12:13:22.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBiW5RpOPycLxAA6LjABDjt5XOWnB1MtRtq2A=="}' 2026-02-17T12:13:22.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 0d363fee-fa04-4413-8561-2e808f5c40d2 -i td/osd-scrub-repair/1/new.json 2026-02-17T12:13:22.960 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:13:22.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T12:13:22.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBiW5RpOPycLxAA6LjABDjt5XOWnB1MtRtq2A== --osd-uuid 0d363fee-fa04-4413-8561-2e808f5c40d2 2026-02-17T12:13:22.997 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:22.986+0000 7f8445c9c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:22.998 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:22.986+0000 7f8445c9c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:22.999 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:22.986+0000 7f8445c9c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:23.000 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:22.990+0000 7f8445c9c8c0 -1 bdev(0x555ddd7b1c00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:13:23.000 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:22.990+0000 7f8445c9c8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T12:13:25.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T12:13:25.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:13:25.278 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:13:25.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:13:25.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:13:25.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:13:25.479 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:13:25.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:13:25.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:13:25.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:13:25.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:13:25.502 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:25.490+0000 7f54f8c4d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:25.504 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:25.494+0000 7f54f8c4d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:25.504 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:25.494+0000 7f54f8c4d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:25.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:13:25.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:13:25.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:13:25.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:13:25.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:13:25.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:25.631 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:13:25.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:13:25.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:25.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:13:25.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:26.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:26.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:26.786 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:13:26.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:13:26.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:26.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:13:26.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:26.993 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:26.982+0000 7f54f8c4d8c0 -1 Falling back to public interface 2026-02-17T12:13:27.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:27.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:27.942 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:13:27.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:13:27.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:27.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:13:28.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:29.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:29.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:29.095 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:13:29.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:13:29.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:29.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:13:29.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:30.002 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:29.990+0000 7f54f8c4d8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:13:30.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:30.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:30.250 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:13:30.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:13:30.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:30.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:13:30.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:31.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:31.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:31.440 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:13:31.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:13:31.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:31.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:13:31.592 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3118096037,v1:127.0.0.1:6811/3118096037] [v2:127.0.0.1:6812/3118096037,v1:127.0.0.1:6813/3118096037] exists,up 0d363fee-fa04-4413-8561-2e808f5c40d2 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:860: unfound_erasure_coded: run_osd td/osd-scrub-repair 2 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:13:31.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:13:31.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:13:31.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:31.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:31.594 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:31.594 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:31.594 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:13:31.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:31.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:31.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:13:31.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:13:31.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:13:31.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:13:31.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:13:31.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:13:31.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:13:31.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:13:31.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:13:31.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:13:31.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:13:31.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T12:13:31.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:13:31.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3e4da944-2c48-4683-b5d7-1c748ad80006 2026-02-17T12:13:31.598 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 3e4da944-2c48-4683-b5d7-1c748ad80006 2026-02-17T12:13:31.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 3e4da944-2c48-4683-b5d7-1c748ad80006' 2026-02-17T12:13:31.599 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:13:31.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBrW5RpfHL+IxAA4vkIfxSnZP0kevOh0qyGgg== 2026-02-17T12:13:31.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBrW5RpfHL+IxAA4vkIfxSnZP0kevOh0qyGgg=="}' 2026-02-17T12:13:31.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3e4da944-2c48-4683-b5d7-1c748ad80006 -i td/osd-scrub-repair/2/new.json 2026-02-17T12:13:31.766 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:13:31.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T12:13:31.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBrW5RpfHL+IxAA4vkIfxSnZP0kevOh0qyGgg== --osd-uuid 3e4da944-2c48-4683-b5d7-1c748ad80006 2026-02-17T12:13:31.801 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:31.790+0000 7f86533278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:31.802 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:31.790+0000 7f86533278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:31.803 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:31.794+0000 7f86533278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:31.804 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:31.794+0000 7f86533278c0 -1 bdev(0x563cc8c7bc00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:13:31.804 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:31.794+0000 7f86533278c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T12:13:34.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T12:13:34.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:13:34.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:13:34.107 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T12:13:34.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:13:34.301 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T12:13:34.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:13:34.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:13:34.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:13:34.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:13:34.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:13:34.317 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:34.306+0000 7f92d9c118c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:34.319 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:34.306+0000 7f92d9c118c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:34.320 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:34.310+0000 7f92d9c118c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:34.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:13:34.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:13:34.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:13:34.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:13:34.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:13:34.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:34.451 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:13:34.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:13:34.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:34.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:13:34.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:35.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:35.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:35.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:13:35.612 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:13:35.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:35.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:13:35.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:35.813 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:35.802+0000 7f92d9c118c0 -1 Falling back to public interface 2026-02-17T12:13:36.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:36.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:36.764 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:13:36.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:13:36.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:36.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:13:36.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:37.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:37.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:37.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:13:37.923 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:13:37.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:37.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:13:38.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:38.852 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:38.838+0000 7f92d9c118c0 -1 osd.2 0 log_to_monitors true 2026-02-17T12:13:39.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:39.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:39.079 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:13:39.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:13:39.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:39.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:13:39.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:40.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:40.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:40.230 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:13:40.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:13:40.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:40.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:13:40.383 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2941938515,v1:127.0.0.1:6819/2941938515] [v2:127.0.0.1:6820/2941938515,v1:127.0.0.1:6821/2941938515] exists,up 3e4da944-2c48-4683-b5d7-1c748ad80006 2026-02-17T12:13:40.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:13:40.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:13:40.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:13:40.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-17T12:13:40.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:860: unfound_erasure_coded: run_osd td/osd-scrub-repair 3 2026-02-17T12:13:40.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:13:40.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:13:40.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:13:40.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:13:40.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T12:13:40.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:13:40.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:13:40.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:13:40.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:13:40.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T12:13:40.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T12:13:40.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:13:40.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:13:40.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:13:40.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:13:40.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:40.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:40.390 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:40.390 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:40.390 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:13:40.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:40.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:40.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:13:40.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:13:40.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:13:40.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:13:40.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:13:40.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:13:40.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:13:40.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:13:40.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:13:40.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:13:40.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:13:40.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T12:13:40.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:13:40.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=30bb19f9-083e-4cab-affc-2869c48eaf9c 2026-02-17T12:13:40.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 30bb19f9-083e-4cab-affc-2869c48eaf9c' 2026-02-17T12:13:40.392 INFO:tasks.workunit.client.0.trial128.stdout:add osd3 30bb19f9-083e-4cab-affc-2869c48eaf9c 2026-02-17T12:13:40.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:13:40.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB0W5RppSZyFxAA0D5EdiIQyPbpz7vJngaSUQ== 2026-02-17T12:13:40.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB0W5RppSZyFxAA0D5EdiIQyPbpz7vJngaSUQ=="}' 2026-02-17T12:13:40.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 30bb19f9-083e-4cab-affc-2869c48eaf9c -i td/osd-scrub-repair/3/new.json 2026-02-17T12:13:40.548 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:13:40.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-17T12:13:40.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB0W5RppSZyFxAA0D5EdiIQyPbpz7vJngaSUQ== --osd-uuid 30bb19f9-083e-4cab-affc-2869c48eaf9c 2026-02-17T12:13:40.575 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:40.562+0000 7f8b8cca28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:40.577 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:40.566+0000 7f8b8cca28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:40.578 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:40.566+0000 7f8b8cca28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:40.578 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:40.566+0000 7f8b8cca28c0 -1 bdev(0x55aeb0abbc00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:13:40.579 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:40.566+0000 7f8b8cca28c0 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-17T12:13:42.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-17T12:13:42.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:13:42.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:13:42.870 INFO:tasks.workunit.client.0.trial128.stdout:adding osd3 key to auth repository 2026-02-17T12:13:42.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:13:43.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:13:43.067 INFO:tasks.workunit.client.0.trial128.stdout:start osd.3 2026-02-17T12:13:43.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:13:43.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:13:43.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:13:43.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:13:43.084 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:43.074+0000 7f2d0dac88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:43.085 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:43.074+0000 7f2d0dac88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:43.086 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:43.074+0000 7f2d0dac88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:13:43.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:13:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:13:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:13:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:13:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:13:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:43.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:13:43.220 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:13:43.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:43.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:13:43.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:44.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:44.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:44.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:13:44.375 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:13:44.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:44.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:13:44.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:44.593 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:44.582+0000 7f2d0dac88c0 -1 Falling back to public interface 2026-02-17T12:13:45.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:45.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:45.530 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:13:45.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:13:45.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:45.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:13:45.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:46.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:46.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:46.682 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:13:46.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:13:46.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:46.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:13:46.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:47.622 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:13:47.610+0000 7f2d0dac88c0 -1 osd.3 0 log_to_monitors true 2026-02-17T12:13:47.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:47.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:47.838 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:13:47.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:13:47.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:47.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:13:47.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:13:48.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:13:48.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:13:48.993 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:13:48.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:13:48.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:13:48.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:13:49.147 INFO:tasks.workunit.client.0.trial128.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2364265034,v1:127.0.0.1:6827/2364265034] [v2:127.0.0.1:6828/2364265034,v1:127.0.0.1:6829/2364265034] exists,up 30bb19f9-083e-4cab-affc-2869c48eaf9c 2026-02-17T12:13:49.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:13:49.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:13:49.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:13:49.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:863: unfound_erasure_coded: create_ec_pool ecpool false k=2 m=2 2026-02-17T12:13:49.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T12:13:49.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T12:13:49.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=false 2026-02-17T12:13:49.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T12:13:49.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=2 2026-02-17T12:13:49.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T12:13:49.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T12:13:49.656 INFO:tasks.workunit.client.0.trial128.stderr:pool 'ecpool' created 2026-02-17T12:13:49.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:13:50.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' false = true ']' 2026-02-17T12:13:50.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T12:13:50.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:13:50.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:13:50.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:13:50.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:13:50.670 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:13:50.670 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:13:50.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:13:50.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:13:50.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:13:50.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:13:50.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:13:50.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:13:50.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:13:50.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:13:50.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:13:50.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:13:50.941 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:13:50.941 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:13:50.941 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T12:13:50.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:13:50.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:50.942 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:13:51.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T12:13:51.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T12:13:51.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T12:13:51.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:51.014 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:13:51.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-17T12:13:51.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-17T12:13:51.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669' 2026-02-17T12:13:51.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:51.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:13:51.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574852 2026-02-17T12:13:51.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574852 2026-02-17T12:13:51.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-55834574852' 2026-02-17T12:13:51.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:13:51.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:13:51.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411330 2026-02-17T12:13:51.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411330 2026-02-17T12:13:51.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-55834574852 3-77309411330' 2026-02-17T12:13:51.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:51.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T12:13:51.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:51.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:13:51.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T12:13:51.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:51.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T12:13:51.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T12:13:51.240 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836487 2026-02-17T12:13:51.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:13:51.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-17T12:13:51.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:13:52.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:13:52.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:13:52.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T12:13:52.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:52.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-17T12:13:52.561 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:52.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:13:52.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-17T12:13:52.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:52.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-17T12:13:52.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-17T12:13:52.566 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705669 2026-02-17T12:13:52.566 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:13:52.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705670 -lt 38654705669 2026-02-17T12:13:52.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:52.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574852 2026-02-17T12:13:52.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:52.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:13:52.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574852 2026-02-17T12:13:52.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:52.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574852 2026-02-17T12:13:52.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574852' 2026-02-17T12:13:52.723 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574852 2026-02-17T12:13:52.723 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:13:52.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-17T12:13:52.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:13:52.888 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-77309411330 2026-02-17T12:13:52.888 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:13:52.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:13:52.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-77309411330 2026-02-17T12:13:52.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:13:52.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411330 2026-02-17T12:13:52.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 77309411330' 2026-02-17T12:13:52.892 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.3 seq 77309411330 2026-02-17T12:13:52.893 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:13:53.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411330 -lt 77309411330 2026-02-17T12:13:53.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:13:53.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:13:53.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:13:53.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:13:53.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:13:53.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:13:53.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:13:53.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:13:53.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:13:53.227 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:13:53.227 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:13:53.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:13:53.377 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:13:53.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:13:53.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:13:53.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:13:53.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:13:53.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:13:53.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T12:13:53.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:865: unfound_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-17T12:13:53.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T12:13:53.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T12:13:53.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T12:13:53.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T12:13:53.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T12:13:53.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T12:13:53.759 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:13:53.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T12:13:53.963 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:13:53.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T12:13:53.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T12:13:53.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T12:13:54.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:867: unfound_erasure_coded: get_primary ecpool SOMETHING 2026-02-17T12:13:54.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T12:13:54.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T12:13:54.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T12:13:54.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:13:54.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:867: unfound_erasure_coded: local primary=1 2026-02-17T12:13:54.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:868: unfound_erasure_coded: get_osds ecpool SOMETHING 2026-02-17T12:13:54.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:868: unfound_erasure_coded: sed -e s/1// 2026-02-17T12:13:54.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T12:13:54.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:13:54.156 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-17T12:13:54.156 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:13:54.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T12:13:54.306 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:13:54.307 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:13:54.307 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T12:13:54.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2 3 2026-02-17T12:13:54.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:868: unfound_erasure_coded: osds=('0' '2' '3') 2026-02-17T12:13:54.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:868: unfound_erasure_coded: local -a osds 2026-02-17T12:13:54.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:869: unfound_erasure_coded: local not_primary_first=0 2026-02-17T12:13:54.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:870: unfound_erasure_coded: local not_primary_second=2 2026-02-17T12:13:54.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:871: unfound_erasure_coded: local not_primary_third=3 2026-02-17T12:13:54.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:876: unfound_erasure_coded: pids= 2026-02-17T12:13:54.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:877: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T12:13:54.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T12:13:54.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T12:13:54.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 436256"' 2026-02-17T12:13:54.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 436256' 2026-02-17T12:13:54.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:878: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T12:13:54.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T12:13:54.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 436257"' 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 436257' 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:879: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/436259: /' 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 436260"' 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 436260' 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:880: unfound_erasure_coded: wait_background pids 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 436256 436257 436260' 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 436256 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/436262: /' 2026-02-17T12:13:54.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T12:13:54.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/436266: /' 2026-02-17T12:13:56.830 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:13:56.831 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:13:56.831 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T12:13:56.831 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:13:56.831 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T12:13:56.831 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:13:56.831 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:13:56.831 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T12:13:56.831 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:13:56.831 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T12:13:56.831 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:13:56.831 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:13:56.831 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:13:56.831 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:13:56.832 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:13:56.832 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:13:56.832 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T12:13:56.832 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:13:56.832 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:13:56.832 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T12:13:56.832 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:13:56.832 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T12:13:56.832 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING remove 2026-02-17T12:13:56.832 INFO:tasks.workunit.client.0.trial128.stderr:436262: remove 2#1:eb822e21:::SOMETHING:head# 2026-02-17T12:13:56.832 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T12:13:56.832 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:13:56.833 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:13:56.833 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:13:56.833 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:13:56.833 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T12:13:56.833 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:13:56.833 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:13:56.835 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:13:56.835 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:13:56.835 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T12:13:56.835 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T12:13:56.835 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:13:56.835 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:13:56.835 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:13:56.835 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:13:56.836 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:56.836 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:56.836 INFO:tasks.workunit.client.0.trial128.stderr:436262: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:56.836 INFO:tasks.workunit.client.0.trial128.stderr:436262: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:56.836 INFO:tasks.workunit.client.0.trial128.stderr:436262: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:13:56.836 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:56.836 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:56.836 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:13:56.836 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:13:56.836 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:13:56.837 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:13:56.837 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:13:56.837 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:13:56.837 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:13:56.837 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:13:56.837 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:13:56.837 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T12:13:56.837 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:13:56.837 INFO:tasks.workunit.client.0.trial128.stderr:436262: start osd.2 2026-02-17T12:13:56.838 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: remove 1#1:eb822e21:::SOMETHING:head# 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:13:56.887 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:13:56.888 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:13:56.888 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:13:56.888 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:13:56.888 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:13:56.891 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:13:56.892 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:13:56.892 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:13:56.892 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:13:56.892 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:13:56.892 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:13:56.892 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:13:56.892 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:13:56.892 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:13:56.892 INFO:tasks.workunit.client.0.trial128.stderr:436259: start osd.0 2026-02-17T12:13:56.892 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:13:57.068 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:13:57.068 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:13:57.068 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T12:13:57.068 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: remove 3#1:eb822e21:::SOMETHING:head# 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:13:57.069 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:13:57.072 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:13:57.072 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:13:57.072 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T12:13:57.072 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T12:13:57.072 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:13:57.072 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:13:57.072 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:13:57.072 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:13:57.072 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:13:57.072 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:13:57.072 INFO:tasks.workunit.client.0.trial128.stderr:436266: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:13:57.072 INFO:tasks.workunit.client.0.trial128.stderr:436266: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:13:57.072 INFO:tasks.workunit.client.0.trial128.stderr:436266: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:13:57.072 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:57.072 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:13:57.072 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:13:57.073 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:13:57.073 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:13:57.073 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:13:57.073 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:13:57.073 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:13:57.073 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:13:57.073 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:13:57.073 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:13:57.073 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T12:13:57.073 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T12:13:57.073 INFO:tasks.workunit.client.0.trial128.stderr:436266: start osd.3 2026-02-17T12:13:57.073 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:14:01.495 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T12:14:01.495 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:14:01.495 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:14:01.495 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:14:01.495 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:14:01.495 INFO:tasks.workunit.client.0.trial128.stderr:436262: 2026-02-17T12:13:56.846+0000 7f611ea7f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:01.495 INFO:tasks.workunit.client.0.trial128.stderr:436262: 2026-02-17T12:13:56.846+0000 7f611ea7f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:01.495 INFO:tasks.workunit.client.0.trial128.stderr:436262: 2026-02-17T12:13:56.850+0000 7f611ea7f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:01.496 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:14:01.496 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:14:01.496 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:14:01.496 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:14:01.496 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:14:01.496 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:01.496 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:14:01.496 INFO:tasks.workunit.client.0.trial128.stderr:436262: 0 2026-02-17T12:14:01.496 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:01.497 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:14:01.497 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:01.497 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:01.497 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:01.497 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:14:01.497 INFO:tasks.workunit.client.0.trial128.stderr:436262: 1 2026-02-17T12:14:01.497 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:01.497 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:14:01.497 INFO:tasks.workunit.client.0.trial128.stderr:436262: 2026-02-17T12:13:58.145+0000 7f611ea7f8c0 -1 Falling back to public interface 2026-02-17T12:14:01.497 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:01.498 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:01.498 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:01.498 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:14:01.498 INFO:tasks.workunit.client.0.trial128.stderr:436262: 2 2026-02-17T12:14:01.498 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:01.498 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:14:01.498 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:01.498 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:01.498 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:01.498 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:14:01.498 INFO:tasks.workunit.client.0.trial128.stderr:436262: 3 2026-02-17T12:14:01.498 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:01.499 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:14:01.499 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:01.544 INFO:tasks.workunit.client.0.trial128.stderr:436262: 2026-02-17T12:14:01.485+0000 7f611ea7f8c0 436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T12:14:01.544 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:14:01.545 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:14:01.545 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:14:01.545 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:14:01.545 INFO:tasks.workunit.client.0.trial128.stderr:436259: 2026-02-17T12:13:56.894+0000 7f595959f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:01.545 INFO:tasks.workunit.client.0.trial128.stderr:436259: 2026-02-17T12:13:56.894+0000 7f595959f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:01.545 INFO:tasks.workunit.client.0.trial128.stderr:436259: 2026-02-17T12:13:56.898+0000 7f595959f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:01.545 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:14:01.545 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:14:01.546 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:14:01.546 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:14:01.546 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:14:01.546 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:01.546 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:14:01.546 INFO:tasks.workunit.client.0.trial128.stderr:436259: 0 2026-02-17T12:14:01.546 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:01.546 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:14:01.546 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:01.546 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:01.547 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:01.547 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:14:01.547 INFO:tasks.workunit.client.0.trial128.stderr:436259: 1 2026-02-17T12:14:01.547 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:01.547 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:14:01.547 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:01.547 INFO:tasks.workunit.client.0.trial128.stderr:436259: 2026-02-17T12:13:58.405+0000 7f595959f8c0 -1 Falling back to public interface 2026-02-17T12:14:01.547 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:01.547 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:01.547 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:14:01.548 INFO:tasks.workunit.client.0.trial128.stderr:436259: 2 2026-02-17T12:14:01.548 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:01.548 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:14:01.548 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:01.548 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:01.548 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:01.548 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:14:01.548 INFO:tasks.workunit.client.0.trial128.stderr:436259: 3 2026-02-17T12:14:01.548 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:01.548 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:14:01.549 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:01.703 INFO:tasks.workunit.client.0.trial128.stderr:436259: 2026-02-17T12:14:01.537+0000 7f595959f8c0 436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T12:14:01.703 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T12:14:01.703 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:14:01.703 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:14:01.704 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:14:01.704 INFO:tasks.workunit.client.0.trial128.stderr:436266: 2026-02-17T12:13:57.074+0000 7fe63f4a78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:01.704 INFO:tasks.workunit.client.0.trial128.stderr:436266: 2026-02-17T12:13:57.074+0000 7fe63f4a78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:01.704 INFO:tasks.workunit.client.0.trial128.stderr:436266: 2026-02-17T12:13:57.078+0000 7fe63f4a78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:01.704 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T12:14:01.704 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:14:01.704 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:14:01.704 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:14:01.705 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:14:01.705 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:01.705 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:14:01.705 INFO:tasks.workunit.client.0.trial128.stderr:436266: 0 2026-02-17T12:14:01.705 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:01.705 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:14:01.705 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:01.705 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:01.705 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:01.706 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:14:01.706 INFO:tasks.workunit.client.0.trial128.stderr:436266: 1 2026-02-17T12:14:01.706 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:01.706 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:14:01.706 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:01.706 INFO:tasks.workunit.client.0.trial128.stderr:436266: 2026-02-17T12:13:58.577+0000 7fe63f4a78c0 -1 Falling back to public interface 2026-02-17T12:14:01.706 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:01.706 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:01.706 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:14:01.706 INFO:tasks.workunit.client.0.trial128.stderr:436266: 2 2026-02-17T12:14:01.706 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:01.707 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:14:01.707 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:01.707 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:01.707 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:01.707 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:14:01.707 INFO:tasks.workunit.client.0.trial128.stderr:436266: 3 2026-02-17T12:14:01.707 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:01.707 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:14:01.707 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436266: 2026-02-17T12:14:01.693+0000 7fe63f4a78c0 -1 osd.2 26 log_to_monitors true 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: 4 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: osd.2 up in weight 1 up_from 30 up_thru 0 down_at 27 last_clean_interval [13,26) [v2:127.0.0.1:6802/2593417390,v1:127.0.0.1:6803/2593417390] [v2:127.0.0.1:6804/2593417390,v1:127.0.0.1:6805/2593417390] exists,up 3e4da944-2c48-4683-b5d7-1c748ad80006 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:14:02.122 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: 1 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: 2 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: 3' 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018882 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018882 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018882' 2026-02-17T12:14:02.123 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-h-1 osd.0 26 log_to_monitors true 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: 4 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: osd.0 up in weight 1 up_from 30 up_thru 0 down_at 27 last_clean_interval [5,26) [v2:127.0.0.1:6818/50010593,v1:127.0.0.1:6819/50010593] [v2:127.0.0.1:6820/50010593,v1:127.0.0.1:6821/50010593] exists,up df05ea12-ead7-4d46-916c-cd2203cc573d 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:14:02.143 INFO:tasks.workunit.client.0.trial128.stderr:436259: 1 2026-02-17T12:14:02.144 INFO:tasks.workunit.client.0.trial128.stderr:436259: 2 2026-02-17T12:14:02.144 INFO:tasks.workunit.client.0.trial128.stderr:436259: 3' 2026-02-17T12:14:02.144 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:14:02.144 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:02.144 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:14:02.144 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018883 2026-02-17T12:14:02.144 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018883 2026-02-17T12:14:02.144 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018883' 2026-02-17T12:14:02.144 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:02.344 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh-1 osd.3 26 log_to_monitors true 2026-02-17T12:14:02.344 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:02.344 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:02.344 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:14:02.344 INFO:tasks.workunit.client.0.trial128.stderr:436266: 4 2026-02-17T12:14:02.344 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:02.344 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:14:02.344 INFO:tasks.workunit.client.0.trial128.stderr:436266: osd.3 up in weight 1 up_from 32 up_thru 0 down_at 27 last_clean_interval [18,26) [v2:127.0.0.1:6826/3178760340,v1:127.0.0.1:6827/3178760340] [v2:127.0.0.1:6828/3178760340,v1:127.0.0.1:6829/3178760340] exists,up 30bb19f9-083e-4cab-affc-2869c48eaf9c 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: 1 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: 2 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: 3' 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018884 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018884 2026-02-17T12:14:02.345 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018884' 2026-02-17T12:14:02.346 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:02.674 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:14:02.675 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705673 2026-02-17T12:14:02.675 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705673 2026-02-17T12:14:02.675 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018882 1-38654705673' 2026-02-17T12:14:02.675 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:02.675 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:14:02.675 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018882 2026-02-17T12:14:02.675 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018882 2026-02-17T12:14:02.675 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018882 1-38654705673 2-128849018882' 2026-02-17T12:14:02.675 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:02.675 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:14:02.676 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953474 2026-02-17T12:14:02.676 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953474 2026-02-17T12:14:02.676 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018882 1-38654705673 2-128849018882 3-137438953474' 2026-02-17T12:14:02.676 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:02.676 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-128849018882 2026-02-17T12:14:02.676 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:02.676 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:14:02.676 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-128849018882 2026-02-17T12:14:02.676 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:02.676 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018882 2026-02-17T12:14:02.676 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 128849018882' 2026-02-17T12:14:02.676 INFO:tasks.workunit.client.0.trial128.stderr:436262: waiting osd.0 seq 128849018882 2026-02-17T12:14:02.677 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:14:02.677 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018883 -lt 128849018882 2026-02-17T12:14:02.677 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:02.677 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705673 2026-02-17T12:14:02.677 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:02.677 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:14:02.677 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705673 2026-02-17T12:14:02.677 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:02.677 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705673 2026-02-17T12:14:02.677 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705673' 2026-02-17T12:14:02.677 INFO:tasks.workunit.client.0.trial128.stderr:436262: waiting osd.1 seq 38654705673 2026-02-17T12:14:02.677 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:14:02.680 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:14:02.680 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705674 2026-02-17T12:14:02.680 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705674 2026-02-17T12:14:02.680 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018883 1-38654705674' 2026-02-17T12:14:02.680 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:02.680 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:14:02.680 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018883 2026-02-17T12:14:02.680 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018883 2026-02-17T12:14:02.681 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018883 1-38654705674 2-128849018883' 2026-02-17T12:14:02.681 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:02.681 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:14:02.681 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953475 2026-02-17T12:14:02.681 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953475 2026-02-17T12:14:02.681 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018883 1-38654705674 2-128849018883 3-137438953475' 2026-02-17T12:14:02.681 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:02.681 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-128849018883 2026-02-17T12:14:02.681 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:02.681 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:14:02.681 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-128849018883 2026-02-17T12:14:02.681 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:02.681 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018883 2026-02-17T12:14:02.681 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 128849018883' 2026-02-17T12:14:02.682 INFO:tasks.workunit.client.0.trial128.stderr:436259: waiting osd.0 seq 128849018883 2026-02-17T12:14:02.682 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:14:02.682 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018883 -lt 128849018883 2026-02-17T12:14:02.682 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:02.682 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705674 2026-02-17T12:14:02.682 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:02.682 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:14:02.682 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705674 2026-02-17T12:14:02.682 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:02.682 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705674 2026-02-17T12:14:02.682 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-17T12:14:02.682 INFO:tasks.workunit.client.0.trial128.stderr:436259: waiting osd.1 seq 38654705674 2026-02-17T12:14:02.683 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:14:05.054 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: fluselpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:14:05.054 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705675 2026-02-17T12:14:05.054 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705675 2026-02-17T12:14:05.055 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018884 1-38654705675' 2026-02-17T12:14:05.055 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:05.055 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:14:05.055 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018884 2026-02-17T12:14:05.055 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018884 2026-02-17T12:14:05.055 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018884 1-38654705675 2-128849018884' 2026-02-17T12:14:05.055 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:05.055 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:14:05.055 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953476 2026-02-17T12:14:05.055 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953476 2026-02-17T12:14:05.055 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-128849018884 1-38654705675 2-128849018884 3-137438953476' 2026-02-17T12:14:05.055 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:05.056 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-128849018884 2026-02-17T12:14:05.056 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:05.056 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:14:05.056 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-128849018884 2026-02-17T12:14:05.056 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:05.056 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018884 2026-02-17T12:14:05.056 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 128849018884' 2026-02-17T12:14:05.056 INFO:tasks.workunit.client.0.trial128.stderr:436266: waiting osd.0 seq 128849018884 2026-02-17T12:14:05.056 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:14:05.056 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018883 -lt 128849018884 2026-02-17T12:14:05.056 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:14:05.056 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:14:05.057 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:14:05.057 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018883 -lt 128849018884 2026-02-17T12:14:05.057 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:14:05.057 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:14:05.057 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:14:05.057 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018884 -lt 128849018884 2026-02-17T12:14:05.057 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:05.519 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stat277: flush_pg_stats: test 38654705674 -lt 38654705673 2026-02-17T12:14:05.519 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:05.519 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-128849018882 2026-02-17T12:14:05.519 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:05.519 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:14:05.520 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-128849018882 2026-02-17T12:14:05.520 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:05.520 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018882 2026-02-17T12:14:05.520 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 128849018882' 2026-02-17T12:14:05.520 INFO:tasks.workunit.client.0.trial128.stderr:436262: waiting osd.2 seq 128849018882 2026-02-17T12:14:05.520 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:14:05.520 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 128849018882 2026-02-17T12:14:05.520 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:14:05.520 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:14:05.520 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:14:05.520 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 128849018882 2026-02-17T12:14:05.520 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:14:05.521 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:14:05.521 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:14:05.521 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018884 -lt 128849018882 2026-02-17T12:14:05.521 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:05.521 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-137438953474 2026-02-17T12:14:05.521 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:05.521 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:14:05.521 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-137438953474 2026-02-17T12:14:05.521 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:05.521 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953474 2026-02-17T12:14:05.521 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 137438953474' 2026-02-17T12:14:05.521 INFO:tasks.workunit.client.0.trial128.stderr:436262: waiting osd.3 seq 137438953474 2026-02-17T12:14:05.522 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:14:05.522 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953476 -lt 137438953474 2026-02-17T12:14:05.522 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:14:05.522 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:14:05.522 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:14:05.522 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:14:05.522 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:14:05.527 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_foh_pg_stats: test 38654705674 -lt 38654705674 2026-02-17T12:14:05.527 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:05.528 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-128849018883 2026-02-17T12:14:05.528 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:05.528 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:14:05.528 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-128849018883 2026-02-17T12:14:05.528 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:05.528 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018883 2026-02-17T12:14:05.528 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 128849018883' 2026-02-17T12:14:05.528 INFO:tasks.workunit.client.0.trial128.stderr:436259: waiting osd.2 seq 128849018883 2026-02-17T12:14:05.528 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:14:05.528 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 128849018883 2026-02-17T12:14:05.528 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:14:05.528 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:14:05.529 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:14:05.529 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 128849018883 2026-02-17T12:14:05.529 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:14:05.529 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:14:05.529 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:14:05.529 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018884 -lt 128849018883 2026-02-17T12:14:05.529 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:05.529 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-137438953475 2026-02-17T12:14:05.529 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:05.529 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:14:05.529 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-137438953475 2026-02-17T12:14:05.529 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:05.529 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953475 2026-02-17T12:14:05.530 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 137438953475' 2026-02-17T12:14:05.530 INFO:tasks.workunit.client.0.trial128.stderr:436259: waiting osd.3 seq 137438953475 2026-02-17T12:14:05.530 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:14:05.530 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953476 -lt 137438953475 2026-02-17T12:14:05.530 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:14:05.530 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:14:05.530 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:14:05.530 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:14:05.530 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:14:05.726 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: s: echo 1-38654705675 2026-02-17T12:14:05.726 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:05.726 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:14:05.726 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705675 2026-02-17T12:14:05.727 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:05.727 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705675 2026-02-17T12:14:05.727 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-17T12:14:05.727 INFO:tasks.workunit.client.0.trial128.stderr:436266: waiting osd.1 seq 38654705675 2026-02-17T12:14:05.727 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:14:05.727 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705675 2026-02-17T12:14:05.727 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:05.727 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-128849018884 2026-02-17T12:14:05.727 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:05.728 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:14:05.728 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-128849018884 2026-02-17T12:14:05.728 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:05.728 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018884 2026-02-17T12:14:05.728 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 128849018884' 2026-02-17T12:14:05.728 INFO:tasks.workunit.client.0.trial128.stderr:436266: waiting osd.2 seq 128849018884 2026-02-17T12:14:05.728 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:14:05.729 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018884 -lt 128849018884 2026-02-17T12:14:05.729 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:05.729 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-137438953476 2026-02-17T12:14:05.729 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:05.729 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:14:05.729 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-137438953476 2026-02-17T12:14:05.730 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:05.730 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953476 2026-02-17T12:14:05.730 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 137438953476' 2026-02-17T12:14:05.730 INFO:tasks.workunit.client.0.trial128.stderr:436266: waiting osd.3 seq 137438953476 2026-02-17T12:14:05.730 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:14:05.730 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953476 -lt 137438953476 2026-02-17T12:14:05.731 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:14:05.731 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:14:05.731 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:14:05.731 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:14:05.731 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:14:05.860 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-her_clean: get_num_active_clean 2026-02-17T12:14:05.860 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:14:05.860 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:14:05.860 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:14:05.860 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:14:05.860 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:14:05.860 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:14:05.860 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:14:05.860 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:14:05.860 INFO:tasks.workunit.client.0.trial128.stderr:436262: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:14:05.860 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:14:05.860 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:14:05.860 INFO:tasks.workunit.client.0.trial128.stderr:436262: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:14:05.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T12:14:05.870 INFO:tasks.workunit.client.0.trial128.stderr: get_num_active_clean 2026-02-17T12:14:05.870 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:14:05.871 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:14:05.871 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:14:05.871 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:14:05.871 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:14:05.871 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:14:05.871 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:14:05.871 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:14:05.872 INFO:tasks.workunit.client.0.trial128.stderr:436259: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:14:05.872 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:14:05.872 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:14:05.872 INFO:tasks.workunit.client.0.trial128.stderr:436259: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:14:05.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T12:14:05.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T12:14:05.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 436257 2026-02-17T12:14:05.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T12:14:05.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 436260 2026-02-17T12:14:06.079 INFO:tasks.workunit.client.0.trial128.stderr:lpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:14:06.079 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:14:06.079 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:14:06.079 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:14:06.079 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:14:06.079 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:14:06.079 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:14:06.079 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:14:06.079 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:14:06.079 INFO:tasks.workunit.client.0.trial128.stderr:436266: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:14:06.079 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:14:06.079 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:14:06.079 INFO:tasks.workunit.client.0.trial128.stderr:436266: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:14:06.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T12:14:06.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T12:14:06.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T12:14:06.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T12:14:06.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:881: unfound_erasure_coded: return_code=0 2026-02-17T12:14:06.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:882: unfound_erasure_coded: '[' 0 -ne 0 ']' 2026-02-17T12:14:06.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:887: unfound_erasure_coded: get_pg ecpool SOMETHING 2026-02-17T12:14:06.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T12:14:06.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T12:14:06.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T12:14:06.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T12:14:06.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:887: unfound_erasure_coded: local pg=1.0 2026-02-17T12:14:06.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:888: unfound_erasure_coded: repair 1.0 2026-02-17T12:14:06.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.0 2026-02-17T12:14:06.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.0 2026-02-17T12:14:06.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:14:06.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:14:06.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:14:06.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:14:06.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T12:13:49.648774+0000 2026-02-17T12:14:06.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.0 2026-02-17T12:14:06.526 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 1.0s0 on osd.1 to repair 2026-02-17T12:14:06.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.0 2026-02-17T12:13:49.648774+0000 2026-02-17T12:14:06.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T12:14:06.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:13:49.648774+0000 2026-02-17T12:14:06.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:14:06.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:14:06.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:14:06.537 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:14:06.537 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:14:06.537 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:14:06.537 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:14:06.537 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:14:06.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:13:49.648774+0000 '>' 2026-02-17T12:13:49.648774+0000 2026-02-17T12:14:06.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:14:07.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:14:07.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:14:07.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:14:07.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:14:07.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:14:07.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:14:07.693 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:14:07.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:13:49.648774+0000 '>' 2026-02-17T12:13:49.648774+0000 2026-02-17T12:14:07.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:14:08.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:14:08.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:14:08.843 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:14:08.843 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:14:08.843 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:14:08.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:14:08.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:14:08.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:13:49.648774+0000 '>' 2026-02-17T12:13:49.648774+0000 2026-02-17T12:14:08.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:14:09.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:14:09.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:14:09.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:14:09.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:14:09.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:14:09.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:14:09.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:14:10.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:13:49.648774+0000 '>' 2026-02-17T12:13:49.648774+0000 2026-02-17T12:14:10.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:14:11.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:14:11.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:14:11.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:14:11.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:14:11.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:14:11.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:14:11.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:14:11.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:13:49.648774+0000 '>' 2026-02-17T12:13:49.648774+0000 2026-02-17T12:14:11.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:14:12.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:14:12.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:14:12.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:14:12.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:14:12.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:14:12.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:14:12.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:14:12.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:14:07.065444+0000 '>' 2026-02-17T12:13:49.648774+0000 2026-02-17T12:14:12.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:14:12.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:893: unfound_erasure_coded: seq 1 60 2026-02-17T12:14:12.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:893: unfound_erasure_coded: for f in `seq 1 60` 2026-02-17T12:14:12.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:894: unfound_erasure_coded: ceph -s 2026-02-17T12:14:12.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:894: unfound_erasure_coded: grep '1/1 objects unfound' 2026-02-17T12:14:12.630 INFO:tasks.workunit.client.0.trial128.stdout: 1/1 objects unfound (100.000%) 2026-02-17T12:14:12.630 INFO:tasks.workunit.client.0.trial128.stdout: 1/1 objects unfound (100.000%) 2026-02-17T12:14:12.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:894: unfound_erasure_coded: break 2026-02-17T12:14:12.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:897: unfound_erasure_coded: ceph -s 2026-02-17T12:14:12.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:897: unfound_erasure_coded: grep '4 up' 2026-02-17T12:14:12.819 INFO:tasks.workunit.client.0.trial128.stdout: osd: 4 osds: 4 up (since 10s), 4 in (since 32s) 2026-02-17T12:14:12.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:898: unfound_erasure_coded: ceph -s 2026-02-17T12:14:12.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:898: unfound_erasure_coded: grep '4 in' 2026-02-17T12:14:13.008 INFO:tasks.workunit.client.0.trial128.stdout: osd: 4 osds: 4 up (since 11s), 4 in (since 32s) 2026-02-17T12:14:13.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:899: unfound_erasure_coded: ceph -s 2026-02-17T12:14:13.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:899: unfound_erasure_coded: grep '1/1 objects unfound' 2026-02-17T12:14:13.198 INFO:tasks.workunit.client.0.trial128.stdout: 1/1 objects unfound (100.000%) 2026-02-17T12:14:13.198 INFO:tasks.workunit.client.0.trial128.stdout: 1/1 objects unfound (100.000%) 2026-02-17T12:14:13.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T12:14:13.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:14:13.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:14:13.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:14:13.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:14:13.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:14:13.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:14:13.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:14:13.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:14:13.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:14:13.311 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:14:13.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:14:13.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:14:13.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:14:13.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:14:13.314 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:14:13.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:14:13.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:14:13.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:14:13.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:14:13.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:14:13.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:14:13.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:14:13.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:14:13.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:13.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:14:13.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:14:13.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:14:13.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:14:13.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:63: run: for func in $funcs 2026-02-17T12:14:13.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:64: run: setup td/osd-scrub-repair 2026-02-17T12:14:13.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-repair 2026-02-17T12:14:13.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-repair 2026-02-17T12:14:13.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:14:13.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:14:13.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:14:13.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:14:13.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:14:13.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:14:13.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:14:13.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:14:13.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:14:13.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:14:13.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:14:13.342 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:14:13.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:14:13.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:14:13.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:14:13.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:14:13.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:14:13.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:14:13.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:14:13.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:14:13.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:14:13.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:14:13.351 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:14:13.351 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:13.351 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:14:13.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:14:13.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:14:13.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:14:13.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-repair 2026-02-17T12:14:13.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:14:13.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:13.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:14:13.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.40886 2026-02-17T12:14:13.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-repair 1' TERM HUP INT 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:65: run: TEST_unfound_erasure_coded_overwrites td/osd-scrub-repair 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:907: TEST_unfound_erasure_coded_overwrites: '[' true = true ']' 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:908: TEST_unfound_erasure_coded_overwrites: unfound_erasure_coded td/osd-scrub-repair true 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:852: unfound_erasure_coded: local dir=td/osd-scrub-repair 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:853: unfound_erasure_coded: local allow_overwrites=true 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:854: unfound_erasure_coded: local poolname=ecpool 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:855: unfound_erasure_coded: local payload=ABCDEF 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:857: unfound_erasure_coded: run_mon td/osd-scrub-repair a 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-repair 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-repair/a 2026-02-17T12:14:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-repair/a --run-dir=td/osd-scrub-repair 2026-02-17T12:14:13.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:14:13.379 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:14:13.379 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:14:13.379 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:14:13.379 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:13.380 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:14:13.380 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:14:13.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-repair/a '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-repair/log --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:14:13.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:14:13.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:14:13.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:14:13.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:14:13.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:14:13.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:14:13.408 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:14:13.408 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:14:13.408 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:14:13.408 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:14:13.408 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:13.408 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:14:13.409 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:14:13.409 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:14:13.409 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get fsid 2026-02-17T12:14:13.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:14:13.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:14:13.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:14:13.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:14:13.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:14:13.466 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:14:13.467 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:14:13.467 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:14:13.467 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:14:13.467 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:13.467 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:14:13.467 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.40886/ceph-mon.a.asok 2026-02-17T12:14:13.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:14:13.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.40886/ceph-mon.a.asok config get mon_host 2026-02-17T12:14:13.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:858: unfound_erasure_coded: run_mgr td/osd-scrub-repair x 2026-02-17T12:14:13.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-repair 2026-02-17T12:14:13.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:14:13.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:14:13.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:14:13.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-repair/x 2026-02-17T12:14:13.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:14:13.614 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:14:13.614 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:14:13.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:14:13.615 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:14:13.615 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:13.615 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:14:13.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:14:13.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:14:13.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-repair/x '--log-file=td/osd-scrub-repair/$name.log' '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --run-dir=td/osd-scrub-repair '--pid-file=td/osd-scrub-repair/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:14:13.636 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: seq 0 3 2026-02-17T12:14:13.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-17T12:14:13.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:860: unfound_erasure_coded: run_osd td/osd-scrub-repair 0 2026-02-17T12:14:13.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:14:13.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:14:13.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:14:13.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:14:13.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:14:13.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:14:13.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:14:13.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:14:13.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:14:13.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:14:13.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:14:13.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:14:13.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:14:13.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:14:13.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:14:13.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:14:13.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:14:13.642 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:14:13.642 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:13.642 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:14:13.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:14:13.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:14:13.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:14:13.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:14:13.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:14:13.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:14:13.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:14:13.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:14:13.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:14:13.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:14:13.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:14:13.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:14:13.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:14:13.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:14:13.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:14:13.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b5065e57-dfbc-438c-9949-fdffa3deb91c 2026-02-17T12:14:13.644 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 b5065e57-dfbc-438c-9949-fdffa3deb91c 2026-02-17T12:14:13.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 b5065e57-dfbc-438c-9949-fdffa3deb91c' 2026-02-17T12:14:13.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:14:13.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCVW5Rp8xCSJhAAcf68PpWxj4Z4sWd8IWe7zw== 2026-02-17T12:14:13.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCVW5Rp8xCSJhAAcf68PpWxj4Z4sWd8IWe7zw=="}' 2026-02-17T12:14:13.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b5065e57-dfbc-438c-9949-fdffa3deb91c -i td/osd-scrub-repair/0/new.json 2026-02-17T12:14:13.745 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:14:13.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/0/new.json 2026-02-17T12:14:13.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCVW5Rp8xCSJhAAcf68PpWxj4Z4sWd8IWe7zw== --osd-uuid b5065e57-dfbc-438c-9949-fdffa3deb91c 2026-02-17T12:14:13.770 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:13.761+0000 7fbe0c5ef8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:13.772 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:13.765+0000 7fbe0c5ef8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:13.772 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:13.765+0000 7fbe0c5ef8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:13.773 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:13.765+0000 7fbe0c5ef8c0 -1 bdev(0x558c526e5c00 td/osd-scrub-repair/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:14:13.773 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:13.765+0000 7fbe0c5ef8c0 -1 bluestore(td/osd-scrub-repair/0) _read_fsid unparsable uuid 2026-02-17T12:14:16.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/0/keyring 2026-02-17T12:14:16.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:14:16.385 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:14:16.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:14:16.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:14:16.579 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:14:16.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:14:16.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:14:16.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:14:16.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:14:16.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:14:16.596 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:16.589+0000 7ffb732dd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:16.597 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:16.589+0000 7ffb732dd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:16.598 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:16.589+0000 7ffb732dd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:16.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:14:16.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:14:16.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:14:16.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:14:16.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:14:16.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:16.736 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:14:16.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:14:16.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:16.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:14:16.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:17.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:17.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:17.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:14:17.892 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:14:17.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:17.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:14:18.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:18.077 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:18.069+0000 7ffb732dd8c0 -1 Falling back to public interface 2026-02-17T12:14:19.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:19.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:19.045 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:14:19.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:14:19.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:19.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:14:19.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:20.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:20.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:20.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:14:20.197 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:14:20.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:20.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:14:20.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:21.059 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:21.049+0000 7ffb732dd8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:14:21.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:21.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:21.354 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:14:21.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:14:21.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:21.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:14:21.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:22.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:22.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:22.509 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:14:22.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:14:22.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:22.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:14:22.658 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2307998243,v1:127.0.0.1:6803/2307998243] [v2:127.0.0.1:6804/2307998243,v1:127.0.0.1:6805/2307998243] exists,up b5065e57-dfbc-438c-9949-fdffa3deb91c 2026-02-17T12:14:22.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:14:22.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:14:22.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:14:22.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-17T12:14:22.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:860: unfound_erasure_coded: run_osd td/osd-scrub-repair 1 2026-02-17T12:14:22.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:14:22.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:14:22.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:14:22.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:14:22.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/1 2026-02-17T12:14:22.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:14:22.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:14:22.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:14:22.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:14:22.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/1' 2026-02-17T12:14:22.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/1/journal' 2026-02-17T12:14:22.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:14:22.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:14:22.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:14:22.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:14:22.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:14:22.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:14:22.660 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:14:22.660 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:22.660 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:14:22.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:14:22.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:14:22.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:14:22.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:14:22.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:14:22.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:14:22.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:14:22.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:14:22.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:14:22.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:14:22.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:14:22.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:14:22.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:14:22.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/1 2026-02-17T12:14:22.663 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:14:22.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=4cf12f15-11f2-4a7e-9da9-8cee5964002d 2026-02-17T12:14:22.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 4cf12f15-11f2-4a7e-9da9-8cee5964002d' 2026-02-17T12:14:22.664 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 4cf12f15-11f2-4a7e-9da9-8cee5964002d 2026-02-17T12:14:22.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:14:22.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCeW5RpAYv2JxAAIyuyOWhmciGx4cJMkRULiQ== 2026-02-17T12:14:22.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCeW5RpAYv2JxAAIyuyOWhmciGx4cJMkRULiQ=="}' 2026-02-17T12:14:22.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 4cf12f15-11f2-4a7e-9da9-8cee5964002d -i td/osd-scrub-repair/1/new.json 2026-02-17T12:14:22.830 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:14:22.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/1/new.json 2026-02-17T12:14:22.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCeW5RpAYv2JxAAIyuyOWhmciGx4cJMkRULiQ== --osd-uuid 4cf12f15-11f2-4a7e-9da9-8cee5964002d 2026-02-17T12:14:22.865 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:22.857+0000 7f662286f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:22.866 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:22.857+0000 7f662286f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:22.867 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:22.861+0000 7f662286f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:22.868 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:22.861+0000 7f662286f8c0 -1 bdev(0x56477b34fc00 td/osd-scrub-repair/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:14:22.868 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:22.861+0000 7f662286f8c0 -1 bluestore(td/osd-scrub-repair/1) _read_fsid unparsable uuid 2026-02-17T12:14:25.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/1/keyring 2026-02-17T12:14:25.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:14:25.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:14:25.140 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:14:25.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:14:25.337 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:14:25.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:14:25.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/1 --osd-journal=td/osd-scrub-repair/1/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:14:25.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:14:25.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:14:25.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:14:25.353 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:25.345+0000 7f2acae548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:25.354 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:25.345+0000 7f2acae548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:25.355 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:25.345+0000 7f2acae548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:25.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:14:25.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:14:25.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:14:25.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:14:25.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:14:25.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:25.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:14:25.493 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:14:25.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:25.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:14:25.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:26.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:26.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:26.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:14:26.644 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:14:26.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:26.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:14:26.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:26.841 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:26.833+0000 7f2acae548c0 -1 Falling back to public interface 2026-02-17T12:14:27.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:27.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:27.797 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:14:27.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:14:27.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:27.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:14:27.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:28.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:28.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:28.949 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:14:28.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:14:28.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:28.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:14:29.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:29.808 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:29.801+0000 7f2acae548c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:14:30.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:30.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:30.101 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:14:30.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:14:30.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:30.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:14:30.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:31.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:31.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:31.257 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:14:31.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:14:31.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:31.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:14:31.408 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/907559086,v1:127.0.0.1:6811/907559086] [v2:127.0.0.1:6812/907559086,v1:127.0.0.1:6813/907559086] exists,up 4cf12f15-11f2-4a7e-9da9-8cee5964002d 2026-02-17T12:14:31.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:14:31.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:14:31.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:14:31.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-17T12:14:31.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:860: unfound_erasure_coded: run_osd td/osd-scrub-repair 2 2026-02-17T12:14:31.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:14:31.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:14:31.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:14:31.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:14:31.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T12:14:31.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:14:31.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:14:31.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:14:31.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:14:31.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T12:14:31.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T12:14:31.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:14:31.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:14:31.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:14:31.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:14:31.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:14:31.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:14:31.414 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:14:31.414 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:31.414 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:14:31.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:14:31.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:14:31.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:14:31.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:14:31.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:14:31.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:14:31.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:14:31.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:14:31.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:14:31.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:14:31.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:14:31.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:14:31.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:14:31.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T12:14:31.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:14:31.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=31f5033b-1cc0-44fe-8d64-9d7300a2607d 2026-02-17T12:14:31.416 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 31f5033b-1cc0-44fe-8d64-9d7300a2607d 2026-02-17T12:14:31.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 31f5033b-1cc0-44fe-8d64-9d7300a2607d' 2026-02-17T12:14:31.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:14:31.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCnW5RpCX0OGRAAzf79I4TGjdAdghEKtk6tig== 2026-02-17T12:14:31.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCnW5RpCX0OGRAAzf79I4TGjdAdghEKtk6tig=="}' 2026-02-17T12:14:31.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 31f5033b-1cc0-44fe-8d64-9d7300a2607d -i td/osd-scrub-repair/2/new.json 2026-02-17T12:14:31.582 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:14:31.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/2/new.json 2026-02-17T12:14:31.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCnW5RpCX0OGRAAzf79I4TGjdAdghEKtk6tig== --osd-uuid 31f5033b-1cc0-44fe-8d64-9d7300a2607d 2026-02-17T12:14:31.608 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:31.601+0000 7f84e3f1e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:31.609 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:31.601+0000 7f84e3f1e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:31.610 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:31.601+0000 7f84e3f1e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:31.611 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:31.601+0000 7f84e3f1e8c0 -1 bdev(0x564716257c00 td/osd-scrub-repair/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:14:31.611 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:31.601+0000 7f84e3f1e8c0 -1 bluestore(td/osd-scrub-repair/2) _read_fsid unparsable uuid 2026-02-17T12:14:33.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/2/keyring 2026-02-17T12:14:33.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:14:33.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:14:33.902 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T12:14:33.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:14:34.089 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T12:14:34.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:14:34.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:14:34.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:14:34.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:14:34.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:14:34.112 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:34.105+0000 7f50bae5a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:34.113 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:34.105+0000 7f50bae5a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:34.114 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:34.105+0000 7f50bae5a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:34.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:14:34.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:14:34.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:14:34.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:14:34.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:14:34.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:34.246 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:14:34.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:14:34.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:34.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:14:34.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:35.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:35.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:35.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:14:35.399 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:14:35.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:35.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:14:35.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:35.605 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:35.597+0000 7f50bae5a8c0 -1 Falling back to public interface 2026-02-17T12:14:36.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:36.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:36.546 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:14:36.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:14:36.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:36.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:14:36.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:37.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:37.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:37.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:14:37.695 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:14:37.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:37.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:14:37.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:38.602 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:38.593+0000 7f50bae5a8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T12:14:38.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:38.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:38.844 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:14:38.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:14:38.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:38.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:14:38.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:39.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:40.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:40.000 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:14:40.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:14:40.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:40.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:14:40.148 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1933500514,v1:127.0.0.1:6819/1933500514] [v2:127.0.0.1:6820/1933500514,v1:127.0.0.1:6821/1933500514] exists,up 31f5033b-1cc0-44fe-8d64-9d7300a2607d 2026-02-17T12:14:40.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:14:40.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:14:40.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:14:40.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:859: unfound_erasure_coded: for id in $(seq 0 3) 2026-02-17T12:14:40.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:860: unfound_erasure_coded: run_osd td/osd-scrub-repair 3 2026-02-17T12:14:40.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-repair 2026-02-17T12:14:40.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:14:40.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-17T12:14:40.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:14:40.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T12:14:40.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:14:40.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:14:40.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:14:40.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:14:40.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T12:14:40.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T12:14:40.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:14:40.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:14:40.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:14:40.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:14:40.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:14:40.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:14:40.152 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:14:40.152 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:40.152 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:14:40.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:14:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:14:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:14:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:14:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:14:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:14:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:14:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:14:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:14:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:14:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:14:40.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:14:40.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:14:40.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T12:14:40.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:14:40.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ee568710-8983-4460-93a1-ca0d3f616d89 2026-02-17T12:14:40.154 INFO:tasks.workunit.client.0.trial128.stdout:add osd3 ee568710-8983-4460-93a1-ca0d3f616d89 2026-02-17T12:14:40.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 ee568710-8983-4460-93a1-ca0d3f616d89' 2026-02-17T12:14:40.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:14:40.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCwW5RpO4mHCRAAtIrY3JVZtKfVMpGtZOLyLQ== 2026-02-17T12:14:40.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCwW5RpO4mHCRAAtIrY3JVZtKfVMpGtZOLyLQ=="}' 2026-02-17T12:14:40.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ee568710-8983-4460-93a1-ca0d3f616d89 -i td/osd-scrub-repair/3/new.json 2026-02-17T12:14:40.320 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:14:40.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-repair/3/new.json 2026-02-17T12:14:40.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCwW5RpO4mHCRAAtIrY3JVZtKfVMpGtZOLyLQ== --osd-uuid ee568710-8983-4460-93a1-ca0d3f616d89 2026-02-17T12:14:40.345 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:40.337+0000 7fd9ca10c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:40.347 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:40.337+0000 7fd9ca10c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:40.348 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:40.341+0000 7fd9ca10c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:40.348 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:40.341+0000 7fd9ca10c8c0 -1 bdev(0x55d501ac9c00 td/osd-scrub-repair/3/block) open stat got: (1) Operation not permitted 2026-02-17T12:14:40.348 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:40.341+0000 7fd9ca10c8c0 -1 bluestore(td/osd-scrub-repair/3) _read_fsid unparsable uuid 2026-02-17T12:14:42.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-repair/3/keyring 2026-02-17T12:14:42.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:14:42.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-17T12:14:42.611 INFO:tasks.workunit.client.0.trial128.stdout:adding osd3 key to auth repository 2026-02-17T12:14:42.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-repair/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:14:42.803 INFO:tasks.workunit.client.0.trial128.stdout:start osd.3 2026-02-17T12:14:42.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-17T12:14:42.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:14:42.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:14:42.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:14:42.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:14:42.826 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:42.817+0000 7f1eb0c618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:42.827 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:42.817+0000 7f1eb0c618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:42.828 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:42.821+0000 7f1eb0c618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:14:42.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-17T12:14:42.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:14:42.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:14:42.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:14:42.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:14:42.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:42.961 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:14:42.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:14:42.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:42.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:14:43.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:44.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:44.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:44.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:14:44.111 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:14:44.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:44.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:14:44.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:44.317 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:44.309+0000 7f1eb0c618c0 -1 Falling back to public interface 2026-02-17T12:14:45.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:45.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:45.257 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:14:45.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:14:45.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:45.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:14:45.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:46.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:46.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:46.407 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:14:46.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:14:46.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:46.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:14:46.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:47.281 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:14:47.273+0000 7f1eb0c618c0 -1 osd.3 0 log_to_monitors true 2026-02-17T12:14:47.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:47.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:47.558 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:14:47.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:14:47.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:47.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:14:47.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:14:48.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:14:48.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:14:48.711 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:14:48.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:14:48.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:14:48.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:14:48.860 INFO:tasks.workunit.client.0.trial128.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2449024152,v1:127.0.0.1:6827/2449024152] [v2:127.0.0.1:6828/2449024152,v1:127.0.0.1:6829/2449024152] exists,up ee568710-8983-4460-93a1-ca0d3f616d89 2026-02-17T12:14:48.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:14:48.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:14:48.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:14:48.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:863: unfound_erasure_coded: create_ec_pool ecpool true k=2 m=2 2026-02-17T12:14:48.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=ecpool 2026-02-17T12:14:48.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-17T12:14:48.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=true 2026-02-17T12:14:48.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-17T12:14:48.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=2 2026-02-17T12:14:49.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool ecpool 1 1 erasure myprofile 2026-02-17T12:14:49.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create ecpool 1 1 erasure myprofile 2026-02-17T12:14:49.367 INFO:tasks.workunit.client.0.trial128.stderr:pool 'ecpool' created 2026-02-17T12:14:49.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:14:50.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' true = true ']' 2026-02-17T12:14:50.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2508: create_ec_pool: ceph osd pool set ecpool allow_ec_overwrites true 2026-02-17T12:14:50.577 INFO:tasks.workunit.client.0.trial128.stderr:set pool 1 allow_ec_overwrites to true 2026-02-17T12:14:50.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-17T12:14:50.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:14:50.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:14:50.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:14:50.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:14:50.589 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:14:50.589 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:14:50.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:14:50.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:14:50.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:14:50.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:14:50.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:14:50.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:14:50.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:14:50.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:14:50.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:14:50.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:14:50.850 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:14:50.850 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:14:50.850 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T12:14:50.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:14:50.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:50.850 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:14:50.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-17T12:14:50.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-17T12:14:50.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-17T12:14:50.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:50.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:14:50.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705670 2026-02-17T12:14:50.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705670 2026-02-17T12:14:50.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705670' 2026-02-17T12:14:50.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:50.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:14:51.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542148 2026-02-17T12:14:51.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542148 2026-02-17T12:14:51.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705670 2-60129542148' 2026-02-17T12:14:51.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:14:51.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:14:51.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=77309411330 2026-02-17T12:14:51.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 77309411330 2026-02-17T12:14:51.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705670 2-60129542148 3-77309411330' 2026-02-17T12:14:51.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:51.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-17T12:14:51.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:51.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:14:51.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-17T12:14:51.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:51.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-17T12:14:51.141 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836487 2026-02-17T12:14:51.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-17T12:14:51.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:14:51.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836486 -lt 21474836487 2026-02-17T12:14:51.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:14:52.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:14:52.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:14:52.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-17T12:14:52.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:52.457 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705670 2026-02-17T12:14:52.457 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:52.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:14:52.460 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705670 2026-02-17T12:14:52.460 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:52.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705670 2026-02-17T12:14:52.462 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705670 2026-02-17T12:14:52.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-17T12:14:52.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:14:52.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705670 -lt 38654705670 2026-02-17T12:14:52.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:52.625 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542148 2026-02-17T12:14:52.625 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:52.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:14:52.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542148 2026-02-17T12:14:52.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:52.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542148 2026-02-17T12:14:52.630 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542148 2026-02-17T12:14:52.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-17T12:14:52.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:14:52.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542148 -lt 60129542148 2026-02-17T12:14:52.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:14:52.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-77309411330 2026-02-17T12:14:52.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:14:52.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:14:52.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-77309411330 2026-02-17T12:14:52.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:14:52.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=77309411330 2026-02-17T12:14:52.796 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.3 seq 77309411330 2026-02-17T12:14:52.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 77309411330' 2026-02-17T12:14:52.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:14:52.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 77309411330 -lt 77309411330 2026-02-17T12:14:52.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:14:52.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:14:52.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:14:53.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:14:53.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:14:53.130 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:14:53.130 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:14:53.130 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:14:53.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:14:53.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:14:53.131 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:14:53.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:14:53.280 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:14:53.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:14:53.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:14:53.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:14:53.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:14:53.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:14:53.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-17T12:14:53.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:865: unfound_erasure_coded: add_something td/osd-scrub-repair ecpool 2026-02-17T12:14:53.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:71: add_something: local dir=td/osd-scrub-repair 2026-02-17T12:14:53.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:72: add_something: local poolname=ecpool 2026-02-17T12:14:53.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:73: add_something: local obj=SOMETHING 2026-02-17T12:14:53.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:74: add_something: local scrub=noscrub 2026-02-17T12:14:53.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:76: add_something: '[' noscrub = noscrub ']' 2026-02-17T12:14:53.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:78: add_something: ceph osd set noscrub 2026-02-17T12:14:53.659 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:14:53.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:79: add_something: ceph osd set nodeep-scrub 2026-02-17T12:14:53.862 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:14:53.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:85: add_something: local payload=ABCDEF 2026-02-17T12:14:53.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:86: add_something: echo ABCDEF 2026-02-17T12:14:53.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:87: add_something: rados --pool ecpool put SOMETHING td/osd-scrub-repair/ORIGINAL 2026-02-17T12:14:53.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:867: unfound_erasure_coded: get_primary ecpool SOMETHING 2026-02-17T12:14:53.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=ecpool 2026-02-17T12:14:53.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T12:14:53.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map ecpool SOMETHING 2026-02-17T12:14:53.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:14:54.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:867: unfound_erasure_coded: local primary=1 2026-02-17T12:14:54.057 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:868: unfound_erasure_coded: get_osds ecpool SOMETHING 2026-02-17T12:14:54.057 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=ecpool 2026-02-17T12:14:54.058 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:868: unfound_erasure_coded: sed -e s/1// 2026-02-17T12:14:54.058 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=SOMETHING 2026-02-17T12:14:54.058 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-17T12:14:54.058 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-17T12:14:54.204 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-17T12:14:54.205 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:14:54.205 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:14:54.205 INFO:tasks.workunit.client.0.trial128.stderr:3' 2026-02-17T12:14:54.205 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2 3 2026-02-17T12:14:54.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:868: unfound_erasure_coded: osds=('0' '2' '3') 2026-02-17T12:14:54.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:868: unfound_erasure_coded: local -a osds 2026-02-17T12:14:54.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:869: unfound_erasure_coded: local not_primary_first=0 2026-02-17T12:14:54.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:870: unfound_erasure_coded: local not_primary_second=2 2026-02-17T12:14:54.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:871: unfound_erasure_coded: local not_primary_third=3 2026-02-17T12:14:54.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:876: unfound_erasure_coded: pids= 2026-02-17T12:14:54.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:877: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T12:14:54.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T12:14:54.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T12:14:54.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 447545"' 2026-02-17T12:14:54.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 447545' 2026-02-17T12:14:54.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:878: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T12:14:54.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T12:14:54.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 447546"' 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 447546' 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:879: unfound_erasure_coded: run_in_background pids objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2190: run_in_background: local pid_variable=pids 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2191: run_in_background: shift 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/447548: /' 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: eval 'pids+=" 447549"' 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2195: run_in_background: pids+=' 447549' 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:880: unfound_erasure_coded: wait_background pids 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2226: wait_background: pids=' 447545 447546 447549' 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2228: wait_background: return_code=0 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 447545 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/447551: /' 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: objectstore_tool td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T12:14:54.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: sed 's/^/447554: /' 2026-02-17T12:14:56.734 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:14:56.734 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:14:56.734 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=3 2026-02-17T12:14:56.734 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:14:56.734 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T12:14:56.735 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:14:56.735 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:14:56.735 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=3 2026-02-17T12:14:56.735 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:14:56.735 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.3 2026-02-17T12:14:56.735 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:14:56.735 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:14:56.735 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:14:56.735 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:14:56.735 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:14:56.735 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:14:56.736 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 3 SOMETHING remove 2026-02-17T12:14:56.736 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:14:56.736 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:14:56.736 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=3 2026-02-17T12:14:56.736 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:14:56.736 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/3 2026-02-17T12:14:56.736 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/3 SOMETHING remove 2026-02-17T12:14:56.736 INFO:tasks.workunit.client.0.trial128.stderr:447554: remove 3#1:eb822e21:::SOMETHING:head# 2026-02-17T12:14:56.736 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 3 2026-02-17T12:14:56.736 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:14:56.736 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:14:56.736 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-17T12:14:56.737 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:14:56.737 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/3 2026-02-17T12:14:56.737 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:14:56.737 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:14:56.737 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:14:56.737 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:14:56.737 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/3' 2026-02-17T12:14:56.737 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/3/journal' 2026-02-17T12:14:56.737 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:14:56.737 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:14:56.737 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:14:56.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:14:56.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:14:56.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:14:56.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:14:56.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:56.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:14:56.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:14:56.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:14:56.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:14:56.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:14:56.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:14:56.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:14:56.739 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:14:56.739 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:14:56.739 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:14:56.739 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:14:56.739 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:14:56.739 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/3 2026-02-17T12:14:56.739 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-17T12:14:56.739 INFO:tasks.workunit.client.0.trial128.stderr:447554: start osd.3 2026-02-17T12:14:56.739 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/3 --osd-journal=td/osd-scrub-repair/3/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:14:56.739 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:14:56.739 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:14:56.739 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=0 2026-02-17T12:14:56.740 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:14:56.740 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T12:14:56.740 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:14:56.740 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:14:56.740 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=0 2026-02-17T12:14:56.740 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:14:56.740 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.0 2026-02-17T12:14:56.740 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:14:56.740 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:14:56.741 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:14:56.741 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:14:56.741 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:14:56.741 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:14:56.741 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 0 SOMETHING remove 2026-02-17T12:14:56.741 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:14:56.741 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:14:56.741 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-17T12:14:56.741 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:14:56.741 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:14:56.741 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/0 SOMETHING remove 2026-02-17T12:14:56.741 INFO:tasks.workunit.client.0.trial128.stderr:447548: remove 1#1:eb822e21:::SOMETHING:head# 2026-02-17T12:14:56.742 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 0 2026-02-17T12:14:56.742 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:14:56.742 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:14:56.742 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:14:56.742 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:14:56.742 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/0 2026-02-17T12:14:56.742 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:14:56.742 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:14:56.743 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:14:56.743 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:14:56.743 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/0' 2026-02-17T12:14:56.743 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/0/journal' 2026-02-17T12:14:56.743 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:14:56.743 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:14:56.743 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:14:56.743 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:14:56.744 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:14:56.744 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:14:56.744 INFO:tasks.workunit.client.0.trial128.stderr:447548: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:14:56.744 INFO:tasks.workunit.client.0.trial128.stderr:447548: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:56.744 INFO:tasks.workunit.client.0.trial128.stderr:447548: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:14:56.744 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:14:56.744 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:14:56.744 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:14:56.744 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:14:56.744 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:14:56.744 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:14:56.744 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:14:56.745 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:14:56.745 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:14:56.745 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:14:56.745 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:14:56.745 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/0 2026-02-17T12:14:56.745 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:14:56.745 INFO:tasks.workunit.client.0.trial128.stderr:447548: start osd.0 2026-02-17T12:14:56.745 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/0 --osd-journal=td/osd-scrub-repair/0/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:14:56.752 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-repair 2026-02-17T12:14:56.752 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:14:56.752 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T12:14:56.752 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:14:56.753 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T12:14:56.753 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-repair 2026-02-17T12:14:56.753 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:14:56.753 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T12:14:56.753 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:14:56.753 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-repair TERM osd.2 2026-02-17T12:14:56.753 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:14:56.753 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:14:56.753 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:14:56.753 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:14:56.753 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:14:56.754 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:14:56.754 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-repair 2 SOMETHING remove 2026-02-17T12:14:56.754 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-repair 2026-02-17T12:14:56.754 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:14:56.754 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T12:14:56.754 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:14:56.754 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-repair/2 2026-02-17T12:14:56.754 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-repair/2 SOMETHING remove 2026-02-17T12:14:56.754 INFO:tasks.workunit.client.0.trial128.stderr:447551: remove 2#1:eb822e21:::SOMETHING:head# 2026-02-17T12:14:56.754 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-repair 2 2026-02-17T12:14:56.754 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-repair 2026-02-17T12:14:56.754 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:14:56.754 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:14:56.755 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:14:56.755 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-repair/2 2026-02-17T12:14:56.755 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false ' 2026-02-17T12:14:56.755 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:14:56.756 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:14:56.756 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:14:56.756 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-repair/2' 2026-02-17T12:14:56.756 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-repair/2/journal' 2026-02-17T12:14:56.756 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:14:56.756 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:14:56.756 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-repair' 2026-02-17T12:14:56.756 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:14:56.756 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:14:56.756 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:14:56.756 INFO:tasks.workunit.client.0.trial128.stderr:447551: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:14:56.757 INFO:tasks.workunit.client.0.trial128.stderr:447551: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:14:56.757 INFO:tasks.workunit.client.0.trial128.stderr:447551: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:14:56.757 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:14:56.757 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' 2026-02-17T12:14:56.757 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:14:56.757 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-repair/$name.log' 2026-02-17T12:14:56.757 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-repair/$name.pid' 2026-02-17T12:14:56.757 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:14:56.757 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:14:56.757 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:14:56.757 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:14:56.757 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:14:56.758 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:14:56.758 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-repair/2 2026-02-17T12:14:56.758 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:14:56.758 INFO:tasks.workunit.client.0.trial128.stderr:447551: start osd.2 2026-02-17T12:14:56.758 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=a7a4e26e-3880-4fb3-94bb-4f9881559052 --auth-supported=none --mon-host=127.0.0.1:7107 --osd-skip-data-digest=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-repair/2 --osd-journal=td/osd-scrub-repair/2/journal --chdir= --run-dir=td/osd-scrub-repair '--admin-socket=/tmp/ceph-asok.40886/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-repair/$name.log' '--pid-file=td/osd-scrub-repair/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:15:01.041 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/3/whoami 2026-02-17T12:15:01.041 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-17T12:15:01.041 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:15:01.041 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:15:01.042 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:15:01.042 INFO:tasks.workunit.client.0.trial128.stderr:447554: 2026-02-17T12:14:56.732+0000 7f6ac529d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:01.042 INFO:tasks.workunit.client.0.trial128.stderr:447554: 2026-02-17T12:14:56.732+0000 7f6ac529d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:01.042 INFO:tasks.workunit.client.0.trial128.stderr:447554: 2026-02-17T12:14:56.732+0000 7f6ac529d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:01.042 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-17T12:15:01.042 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:15:01.042 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-17T12:15:01.042 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:15:01.042 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:15:01.042 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:01.042 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:15:01.043 INFO:tasks.workunit.client.0.trial128.stderr:447554: 0 2026-02-17T12:15:01.043 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:01.043 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:15:01.043 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:01.043 INFO:tasks.workunit.client.0.trial128.stderr:447554: 2026-02-17T12:14:58.012+0000 7f6ac529d8c0 -1 Falling back to public interface 2026-02-17T12:15:01.043 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:01.043 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:01.043 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:15:01.043 INFO:tasks.workunit.client.0.trial128.stderr:447554: 1 2026-02-17T12:15:01.043 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:01.043 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:15:01.043 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:01.043 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:01.044 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:01.044 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:15:01.044 INFO:tasks.workunit.client.0.trial128.stderr:447554: 2 2026-02-17T12:15:01.044 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:01.044 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:15:01.044 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:01.044 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:01.044 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:01.044 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:15:01.044 INFO:tasks.workunit.client.0.trial128.stderr:447554: 3 2026-02-17T12:15:01.044 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:01.044 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:15:01.045 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:01.101 INFO:tasks.workunit.client.0.trial128.stderr:447554: 2026-02-17T12:15:01.032+0000 7f6ac529d8c0 447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/2/whoami 2026-02-17T12:15:01.101 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:15:01.101 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:15:01.101 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:15:01.101 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:15:01.101 INFO:tasks.workunit.client.0.trial128.stderr:447551: 2026-02-17T12:14:56.760+0000 7f4b8fc858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:01.102 INFO:tasks.workunit.client.0.trial128.stderr:447551: 2026-02-17T12:14:56.760+0000 7f4b8fc858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:01.102 INFO:tasks.workunit.client.0.trial128.stderr:447551: 2026-02-17T12:14:56.760+0000 7f4b8fc858c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:01.102 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:15:01.102 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:15:01.102 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:15:01.102 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:15:01.102 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:15:01.102 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:01.102 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:15:01.102 INFO:tasks.workunit.client.0.trial128.stderr:447551: 0 2026-02-17T12:15:01.102 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:01.102 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:15:01.103 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:01.103 INFO:tasks.workunit.client.0.trial128.stderr:447551: 2026-02-17T12:14:58.036+0000 7f4b8fc858c0 -1 Falling back to public interface 2026-02-17T12:15:01.103 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:01.103 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:01.103 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:15:01.103 INFO:tasks.workunit.client.0.trial128.stderr:447551: 1 2026-02-17T12:15:01.103 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:01.103 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:15:01.103 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:01.103 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:01.103 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:01.103 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:15:01.104 INFO:tasks.workunit.client.0.trial128.stderr:447551: 2 2026-02-17T12:15:01.104 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:01.104 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:15:01.104 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:01.104 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:01.104 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:01.104 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:15:01.104 INFO:tasks.workunit.client.0.trial128.stderr:447551: 3 2026-02-17T12:15:01.104 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:01.104 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:15:01.104 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:01.334 INFO:tasks.workunit.client.0.trial128.stderr:447551: 2026-02-17T12:15:01.092+0000 7f4b8fc858c0 447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-repair/0/whoami 2026-02-17T12:15:01.335 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:15:01.335 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:15:01.335 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:15:01.335 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:15:01.336 INFO:tasks.workunit.client.0.trial128.stderr:447548: 2026-02-17T12:14:56.740+0000 7fa65975e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:01.336 INFO:tasks.workunit.client.0.trial128.stderr:447548: 2026-02-17T12:14:56.740+0000 7fa65975e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:01.336 INFO:tasks.workunit.client.0.trial128.stderr:447548: 2026-02-17T12:14:56.744+0000 7fa65975e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:01.336 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:15:01.336 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:15:01.336 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:15:01.337 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:15:01.337 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:15:01.337 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:01.337 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:15:01.337 INFO:tasks.workunit.client.0.trial128.stderr:447548: 0 2026-02-17T12:15:01.337 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:01.337 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:15:01.338 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:01.338 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:01.338 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:01.338 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:15:01.338 INFO:tasks.workunit.client.0.trial128.stderr:447548: 1 2026-02-17T12:15:01.338 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:01.338 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:15:01.338 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:01.339 INFO:tasks.workunit.client.0.trial128.stderr:447548: 2026-02-17T12:14:58.256+0000 7fa65975e8c0 -1 Falling back to public interface 2026-02-17T12:15:01.339 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:01.339 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:01.339 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:15:01.339 INFO:tasks.workunit.client.0.trial128.stderr:447548: 2 2026-02-17T12:15:01.339 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:01.340 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:15:01.340 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:01.340 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:01.340 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:01.340 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:15:01.340 INFO:tasks.workunit.client.0.trial128.stderr:447548: 3 2026-02-17T12:15:01.341 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:01.341 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:15:01.341 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: 2026-02-17T12:15:01.324+0000 7fa65975e8c0 -1 osd.0 27 log_to_monitors true 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: 4 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: osd.0 up in weight 1 up_from 34 up_thru 0 down_at 28 last_clean_interval [5,27) [v2:127.0.0.1:6826/570572512,v1:127.0.0.1:6827/570572512] [v2:127.0.0.1:6828/570572512,v1:127.0.0.1:6829/570572512] exists,up b5065e57-dfbc-438c-9949-fdffa3deb91c 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:15:02.007 INFO:tasks.workunit.client.0.trial128.stderr:447548: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: 1 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: 2 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: 3' 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888066 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888066 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888066' 2026-02-17T12:15:02.008 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helper-1 osd.2 27 log_to_monitors true 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: 4 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: osd.2 up in weight 1 up_from 32 up_thru 0 down_at 28 last_clean_interval [14,27) [v2:127.0.0.1:6818/37720846,v1:127.0.0.1:6819/37720846] [v2:127.0.0.1:6820/37720846,v1:127.0.0.1:6821/37720846] exists,up 31f5033b-1cc0-44fe-8d64-9d7300a2607d 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:15:02.009 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: 1 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: 2 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: 3' 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888067 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888067 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888067' 2026-02-17T12:15:02.010 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:02.034 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.s-1 osd.3 27 log_to_monitors true 2026-02-17T12:15:02.034 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:02.034 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:02.034 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: 4 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: osd.3 up in weight 1 up_from 31 up_thru 0 down_at 28 last_clean_interval [18,27) [v2:127.0.0.1:6802/3619151431,v1:127.0.0.1:6803/3619151431] [v2:127.0.0.1:6804/3619151431,v1:127.0.0.1:6805/3619151431] exists,up ee568710-8983-4460-93a1-ca0d3f616d89 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: ///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:15:02.035 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:15:02.036 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:15:02.036 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:15:02.036 INFO:tasks.workunit.client.0.trial128.stderr:447554: 1 2026-02-17T12:15:02.036 INFO:tasks.workunit.client.0.trial128.stderr:447554: 2 2026-02-17T12:15:02.036 INFO:tasks.workunit.client.0.trial128.stderr:447554: 3' 2026-02-17T12:15:02.036 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:15:02.036 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:02.036 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:15:02.036 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=146028888068 2026-02-17T12:15:02.036 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 146028888068 2026-02-17T12:15:02.036 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888068' 2026-02-17T12:15:02.036 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:04.704 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-hh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:15:04.704 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705674 2026-02-17T12:15:04.705 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705674 2026-02-17T12:15:04.705 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888067 1-38654705674' 2026-02-17T12:15:04.705 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:04.705 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:15:04.705 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953475 2026-02-17T12:15:04.705 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953475 2026-02-17T12:15:04.705 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888067 1-38654705674 2-137438953475' 2026-02-17T12:15:04.705 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:04.705 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:15:04.705 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986178 2026-02-17T12:15:04.705 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986178 2026-02-17T12:15:04.706 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888067 1-38654705674 2-137438953475 3-133143986178' 2026-02-17T12:15:04.706 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:04.706 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-146028888067 2026-02-17T12:15:04.706 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:04.706 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:15:04.706 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-146028888067 2026-02-17T12:15:04.706 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:04.706 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888067 2026-02-17T12:15:04.706 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 146028888067' 2026-02-17T12:15:04.706 INFO:tasks.workunit.client.0.trial128.stderr:447551: waiting osd.0 seq 146028888067 2026-02-17T12:15:04.706 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:15:04.706 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 146028888067 2026-02-17T12:15:04.707 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:15:04.707 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:15:04.707 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:15:04.707 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 146028888067 2026-02-17T12:15:04.707 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:15:04.707 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:15:04.707 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:15:04.707 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888068 -lt 146028888067 2026-02-17T12:15:04.707 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:04.707 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705674 2026-02-17T12:15:04.709 INFO:tasks.workunit.client.0.trial128.stderr:447551:s.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:15:04.709 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705673 2026-02-17T12:15:04.709 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705673 2026-02-17T12:15:04.709 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888066 1-38654705673' 2026-02-17T12:15:04.710 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:04.710 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:15:04.710 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953474 2026-02-17T12:15:04.710 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953474 2026-02-17T12:15:04.710 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888066 1-38654705673 2-137438953474' 2026-02-17T12:15:04.710 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:04.710 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:15:04.710 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986179 2026-02-17T12:15:04.710 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986179 2026-02-17T12:15:04.710 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888066 1-38654705673 2-137438953474 3-133143986179' 2026-02-17T12:15:04.710 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:04.710 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-146028888066 2026-02-17T12:15:04.711 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:04.711 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:15:04.711 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-146028888066 2026-02-17T12:15:04.711 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:04.711 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888066 2026-02-17T12:15:04.711 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 146028888066' 2026-02-17T12:15:04.711 INFO:tasks.workunit.client.0.trial128.stderr:447548: waiting osd.0 seq 146028888066 2026-02-17T12:15:04.711 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:15:04.711 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 146028888066 2026-02-17T12:15:04.711 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:15:04.711 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:15:04.711 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:15:04.712 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 146028888066 2026-02-17T12:15:04.712 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:15:04.712 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:15:04.712 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:15:04.712 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888068 -lt 146028888066 2026-02-17T12:15:04.712 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:04.712 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705673 2026-02-17T12:15:04.713 INFO:tasks.workunit.client.0.trial128.stderr:4475elpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:15:04.713 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705675 2026-02-17T12:15:04.713 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705675 2026-02-17T12:15:04.713 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888068 1-38654705675' 2026-02-17T12:15:04.713 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:04.713 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:15:04.713 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953476 2026-02-17T12:15:04.713 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953476 2026-02-17T12:15:04.713 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888068 1-38654705675 2-137438953476' 2026-02-17T12:15:04.713 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:04.713 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-17T12:15:04.714 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986180 2026-02-17T12:15:04.714 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986180 2026-02-17T12:15:04.714 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-146028888068 1-38654705675 2-137438953476 3-133143986180' 2026-02-17T12:15:04.714 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:04.714 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-146028888068 2026-02-17T12:15:04.714 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:04.714 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:15:04.714 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-146028888068 2026-02-17T12:15:04.714 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:04.714 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=146028888068 2026-02-17T12:15:04.714 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 146028888068' 2026-02-17T12:15:04.714 INFO:tasks.workunit.client.0.trial128.stderr:447554: waiting osd.0 seq 146028888068 2026-02-17T12:15:04.715 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:15:04.715 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 146028888068 2026-02-17T12:15:04.715 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:15:04.715 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:15:04.715 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:15:04.715 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 146028888068 2026-02-17T12:15:04.715 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:15:04.715 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:15:04.715 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:15:04.715 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 146028888068 -lt 146028888068 2026-02-17T12:15:04.715 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:05.377 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-3865470567548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:05.377 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:15:05.377 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705673 2026-02-17T12:15:05.378 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:05.378 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705673 2026-02-17T12:15:05.378 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705673' 2026-02-17T12:15:05.378 INFO:tasks.workunit.client.0.trial128.stderr:447548: waiting osd.1 seq 38654705673 2026-02-17T12:15:05.378 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:15:05.378 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705673 2026-02-17T12:15:05.378 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:05.378 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-137438953474 2026-02-17T12:15:05.378 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:05.378 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:15:05.378 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-137438953474 2026-02-17T12:15:05.378 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:05.379 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953474 2026-02-17T12:15:05.379 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 137438953474' 2026-02-17T12:15:05.379 INFO:tasks.workunit.client.0.trial128.stderr:447548: waiting osd.2 seq 137438953474 2026-02-17T12:15:05.379 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:15:05.379 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953476 -lt 137438953474 2026-02-17T12:15:05.379 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:05.379 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-133143986179 2026-02-17T12:15:05.379 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:05.379 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:15:05.379 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-133143986179 2026-02-17T12:15:05.379 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:05.379 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986179 2026-02-17T12:15:05.380 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 133143986179' 2026-02-17T12:15:05.380 INFO:tasks.workunit.client.0.trial128.stderr:447548: waiting osd.3 seq 133143986179 2026-02-17T12:15:05.380 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:15:05.380 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986180 -lt 133143986179 2026-02-17T12:15:05.380 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:15:05.380 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:15:05.380 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:15:05.380 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:15:05.380 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:15:05.390 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_cle 2026-02-17T12:15:05.390 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:05.390 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:15:05.390 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705675 2026-02-17T12:15:05.390 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:05.390 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705675 2026-02-17T12:15:05.390 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-17T12:15:05.390 INFO:tasks.workunit.client.0.trial128.stderr:447554: waiting osd.1 seq 38654705675 2026-02-17T12:15:05.391 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:15:05.391 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705675 2026-02-17T12:15:05.391 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:05.391 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-137438953476 2026-02-17T12:15:05.391 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:05.391 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:15:05.391 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-137438953476 2026-02-17T12:15:05.391 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:05.391 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953476 2026-02-17T12:15:05.391 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 137438953476' 2026-02-17T12:15:05.391 INFO:tasks.workunit.client.0.trial128.stderr:447554: waiting osd.2 seq 137438953476 2026-02-17T12:15:05.391 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:15:05.392 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953476 -lt 137438953476 2026-02-17T12:15:05.392 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:05.392 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-133143986180 2026-02-17T12:15:05.392 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:05.392 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:15:05.392 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-133143986180 2026-02-17T12:15:05.392 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:05.392 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986180 2026-02-17T12:15:05.392 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 133143986180' 2026-02-17T12:15:05.392 INFO:tasks.workunit.client.0.trial128.stderr:447554: waiting osd.3 seq 133143986180 2026-02-17T12:15:05.392 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:15:05.392 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986180 -lt 133143986180 2026-02-17T12:15:05.392 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:15:05.392 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:15:05.393 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:15:05.393 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:15:05.393 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:15:05.393 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_fo //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:05.393 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:15:05.394 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705674 2026-02-17T12:15:05.394 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:05.394 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705674 2026-02-17T12:15:05.394 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-17T12:15:05.394 INFO:tasks.workunit.client.0.trial128.stderr:447551: waiting osd.1 seq 38654705674 2026-02-17T12:15:05.394 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:15:05.394 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705674 2026-02-17T12:15:05.394 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:05.394 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-137438953475 2026-02-17T12:15:05.394 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:05.394 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:15:05.395 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-137438953475 2026-02-17T12:15:05.395 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:05.395 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953475 2026-02-17T12:15:05.395 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 137438953475' 2026-02-17T12:15:05.395 INFO:tasks.workunit.client.0.trial128.stderr:447551: waiting osd.2 seq 137438953475 2026-02-17T12:15:05.395 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:15:05.395 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953476 -lt 137438953475 2026-02-17T12:15:05.395 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:05.395 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-133143986178 2026-02-17T12:15:05.395 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:05.395 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-17T12:15:05.395 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-133143986178 2026-02-17T12:15:05.395 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986178 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 133143986178' 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial128.stderr:447551: waiting osd.3 seq 133143986178 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986180 -lt 133143986178 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:15:05.396 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:15:05.715 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean:an: get_num_active_clean 2026-02-17T12:15:05.715 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:15:05.715 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:15:05.715 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:15:05.715 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:15:05.715 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:15:05.715 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:15:05.715 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:15:05.715 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:15:05.715 INFO:tasks.workunit.client.0.trial128.stderr:447548: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:15:05.715 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:15:05.715 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:15:05.715 INFO:tasks.workunit.client.0.trial128.stderr:447548: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:15:05.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T12:15:05.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T12:15:05.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 447546 2026-02-17T12:15:05.724 INFO:tasks.workunit.client.0.trial128.stderr: get_num_active_clean 2026-02-17T12:15:05.724 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:15:05.724 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:15:05.724 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:15:05.724 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:15:05.724 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:15:05.724 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:15:05.724 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:15:05.724 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:15:05.724 INFO:tasks.workunit.client.0.trial128.stderr:447551: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:15:05.724 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:15:05.724 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:15:05.724 INFO:tasks.workunit.client.0.trial128.stderr:447551: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:15:05.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T12:15:05.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2229: wait_background: for pid in $pids 2026-02-17T12:15:05.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2230: wait_background: wait 447549 2026-02-17T12:15:05.737 INFO:tasks.workunit.client.0.trial128.stderr:r_clean: get_num_active_clean 2026-02-17T12:15:05.737 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:15:05.737 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:15:05.737 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:15:05.737 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:15:05.737 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:15:05.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:15:05.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:15:05.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:15:05.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: //home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:15:05.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:15:05.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:15:05.738 INFO:tasks.workunit.client.0.trial128.stderr:447554: /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:15:05.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2194: run_in_background: return 0 2026-02-17T12:15:05.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: eval 'pids='\'''\''' 2026-02-17T12:15:05.740 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2237: wait_background: pids= 2026-02-17T12:15:05.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2239: wait_background: return 0 2026-02-17T12:15:05.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:881: unfound_erasure_coded: return_code=0 2026-02-17T12:15:05.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:882: unfound_erasure_coded: '[' 0 -ne 0 ']' 2026-02-17T12:15:05.740 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:887: unfound_erasure_coded: get_pg ecpool SOMETHING 2026-02-17T12:15:05.740 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=ecpool 2026-02-17T12:15:05.741 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T12:15:05.741 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map ecpool SOMETHING 2026-02-17T12:15:05.741 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T12:15:05.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:887: unfound_erasure_coded: local pg=1.0 2026-02-17T12:15:05.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:888: unfound_erasure_coded: repair 1.0 2026-02-17T12:15:05.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.0 2026-02-17T12:15:05.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.0 2026-02-17T12:15:05.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:15:05.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:15:05.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:15:05.895 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:15:06.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T12:14:49.359903+0000 2026-02-17T12:15:06.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.0 2026-02-17T12:15:06.191 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 1.0s0 on osd.1 to repair 2026-02-17T12:15:06.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.0 2026-02-17T12:14:49.359903+0000 2026-02-17T12:15:06.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T12:15:06.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:14:49.359903+0000 2026-02-17T12:15:06.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:15:06.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:15:06.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:15:06.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:15:06.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:15:06.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:15:06.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:15:06.202 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:15:06.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:14:49.359903+0000 '>' 2026-02-17T12:14:49.359903+0000 2026-02-17T12:15:06.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:15:07.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:15:07.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:15:07.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:15:07.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:15:07.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:15:07.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:15:07.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:15:07.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:14:49.359903+0000 '>' 2026-02-17T12:14:49.359903+0000 2026-02-17T12:15:07.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:15:08.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:15:08.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:15:08.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:15:08.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:15:08.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:15:08.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:15:08.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:15:08.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:14:49.359903+0000 '>' 2026-02-17T12:14:49.359903+0000 2026-02-17T12:15:08.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:15:09.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:15:09.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:15:09.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:15:09.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:15:09.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:15:09.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:15:09.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:15:09.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:14:49.359903+0000 '>' 2026-02-17T12:14:49.359903+0000 2026-02-17T12:15:09.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:15:10.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:15:10.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:15:10.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:15:10.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:15:10.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:15:10.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:15:10.832 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:15:10.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:14:49.359903+0000 '>' 2026-02-17T12:14:49.359903+0000 2026-02-17T12:15:10.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:15:11.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:15:11.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:15:11.986 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:15:11.986 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:15:11.986 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:15:11.987 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:15:11.987 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:15:12.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:15:06.841747+0000 '>' 2026-02-17T12:14:49.359903+0000 2026-02-17T12:15:12.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:15:12.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:893: unfound_erasure_coded: seq 1 60 2026-02-17T12:15:12.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:893: unfound_erasure_coded: for f in `seq 1 60` 2026-02-17T12:15:12.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:894: unfound_erasure_coded: ceph -s 2026-02-17T12:15:12.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:894: unfound_erasure_coded: grep '1/1 objects unfound' 2026-02-17T12:15:12.327 INFO:tasks.workunit.client.0.trial128.stdout: 1/1 objects unfound (100.000%) 2026-02-17T12:15:12.327 INFO:tasks.workunit.client.0.trial128.stdout: 1/1 objects unfound (100.000%) 2026-02-17T12:15:12.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:894: unfound_erasure_coded: break 2026-02-17T12:15:12.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:897: unfound_erasure_coded: ceph -s 2026-02-17T12:15:12.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:897: unfound_erasure_coded: grep '4 up' 2026-02-17T12:15:12.514 INFO:tasks.workunit.client.0.trial128.stdout: osd: 4 osds: 4 up (since 11s), 4 in (since 32s) 2026-02-17T12:15:12.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:898: unfound_erasure_coded: ceph -s 2026-02-17T12:15:12.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:898: unfound_erasure_coded: grep '4 in' 2026-02-17T12:15:12.697 INFO:tasks.workunit.client.0.trial128.stdout: osd: 4 osds: 4 up (since 11s), 4 in (since 32s) 2026-02-17T12:15:12.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:899: unfound_erasure_coded: ceph -s 2026-02-17T12:15:12.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:899: unfound_erasure_coded: grep '1/1 objects unfound' 2026-02-17T12:15:12.883 INFO:tasks.workunit.client.0.trial128.stdout: 1/1 objects unfound (100.000%) 2026-02-17T12:15:12.883 INFO:tasks.workunit.client.0.trial128.stdout: 1/1 objects unfound (100.000%) 2026-02-17T12:15:12.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh:66: run: teardown td/osd-scrub-repair 2026-02-17T12:15:12.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:15:12.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:15:12.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:15:12.884 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:15:12.884 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:15:12.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:15:12.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:15:12.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:15:13.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:15:13.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:15:13.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:15:13.201 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:15:13.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:15:13.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:15:13.203 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:15:13.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:15:13.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:15:13.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:15:13.205 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:15:13.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:15:13.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:15:13.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:15:13.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:15:13.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:13.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:15:13.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:15:13.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:15:13.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:15:13.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T12:15:13.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-scrub-repair 0 2026-02-17T12:15:13.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-repair 2026-02-17T12:15:13.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T12:15:13.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-repair KILL 2026-02-17T12:15:13.225 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:15:13.225 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:15:13.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:15:13.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:15:13.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:15:13.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:15:13.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:15:13.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:15:13.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:15:13.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:15:13.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:15:13.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:15:13.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:15:13.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:15:13.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:15:13.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:15:13.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:15:13.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T12:15:13.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-repair 2026-02-17T12:15:13.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:15:13.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:13.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.40886 2026-02-17T12:15:13.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.40886 2026-02-17T12:15:13.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:15:13.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:15:13.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T12:15:13.241 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T12:15:13.241 DEBUG:teuthology.orchestra.run.trial128:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T12:15:13.292 INFO:tasks.workunit:Running workunit scrub/osd-scrub-snaps.sh... 2026-02-17T12:15:13.292 DEBUG:teuthology.orchestra.run.trial128:workunit test scrub/osd-scrub-snaps.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3c96dfb3f50b3d02598a82af7104764bfdcd3919 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh 2026-02-17T12:15:13.345 INFO:tasks.workunit.client.0.trial128.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T12:15:13.349 INFO:tasks.workunit.client.0.trial128.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T12:15:13.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T12:15:13.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T12:15:13.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T12:15:13.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T12:15:13.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T12:15:13.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T12:15:13.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T12:15:13.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T12:15:13.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-scrub-snaps 2026-02-17T12:15:13.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:27: run: local dir=td/osd-scrub-snaps 2026-02-17T12:15:13.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:28: run: shift 2026-02-17T12:15:13.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:30: run: export CEPH_MON=127.0.0.1:7121 2026-02-17T12:15:13.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:30: run: CEPH_MON=127.0.0.1:7121 2026-02-17T12:15:13.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:31: run: export CEPH_ARGS 2026-02-17T12:15:13.350 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:32: run: uuidgen 2026-02-17T12:15:13.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:32: run: CEPH_ARGS+='--fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none ' 2026-02-17T12:15:13.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:33: run: CEPH_ARGS+='--mon-host=127.0.0.1:7121 ' 2026-02-17T12:15:13.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:35: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-17T12:15:13.351 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:36: run: set 2026-02-17T12:15:13.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:36: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T12:15:13.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:36: run: local 'funcs=TEST_scrub_snaps 2026-02-17T12:15:13.354 INFO:tasks.workunit.client.0.trial128.stderr:TEST_scrub_snaps_primary 2026-02-17T12:15:13.354 INFO:tasks.workunit.client.0.trial128.stderr:TEST_scrub_snaps_replica' 2026-02-17T12:15:13.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:37: run: for func in $funcs 2026-02-17T12:15:13.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:38: run: setup td/osd-scrub-snaps 2026-02-17T12:15:13.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-snaps 2026-02-17T12:15:13.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-snaps 2026-02-17T12:15:13.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-snaps 2026-02-17T12:15:13.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:15:13.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-17T12:15:13.355 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:15:13.355 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:15:13.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:15:13.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:15:13.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:15:13.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:15:13.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:15:13.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:15:13.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:15:13.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:15:13.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:15:13.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:15:13.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:15:13.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:15:13.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:15:13.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:15:13.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:15:13.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:15:13.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-snaps 2026-02-17T12:15:13.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:15:13.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:13.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:15:13.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.452141 2026-02-17T12:15:13.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:15:13.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:15:13.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-snaps 2026-02-17T12:15:13.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:15:13.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:13.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:15:13.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.452141 2026-02-17T12:15:13.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:15:13.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T12:15:13.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T12:15:13.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:15:13.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-snaps 1' TERM HUP INT 2026-02-17T12:15:13.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:39: run: TEST_scrub_snaps td/osd-scrub-snaps 2026-02-17T12:15:13.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:171: TEST_scrub_snaps: local dir=td/osd-scrub-snaps 2026-02-17T12:15:13.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:172: TEST_scrub_snaps: local poolname=test 2026-02-17T12:15:13.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:173: TEST_scrub_snaps: local OBJS=16 2026-02-17T12:15:13.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:174: TEST_scrub_snaps: local OSDS=1 2026-02-17T12:15:13.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:176: TEST_scrub_snaps: TESTDATA=testdata.452141 2026-02-17T12:15:13.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:178: TEST_scrub_snaps: run_mon td/osd-scrub-snaps a --osd_pool_default_size=1 2026-02-17T12:15:13.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-snaps 2026-02-17T12:15:13.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:15:13.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:15:13.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:15:13.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-snaps/a 2026-02-17T12:15:13.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-snaps/a --run-dir=td/osd-scrub-snaps --osd_pool_default_size=1 2026-02-17T12:15:13.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:15:13.393 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:15:13.393 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:15:13.393 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:15:13.393 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:13.393 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:15:13.393 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:15:13.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-snaps/a '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-snaps/log --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 2026-02-17T12:15:13.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:15:13.424 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:15:13.424 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:15:13.424 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:15:13.425 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:15:13.425 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:15:13.425 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:15:13.425 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:15:13.425 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:15:13.426 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:15:13.426 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:13.426 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:15:13.426 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.452141/ceph-mon.a.asok 2026-02-17T12:15:13.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:15:13.427 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.452141/ceph-mon.a.asok config get fsid 2026-02-17T12:15:13.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:15:13.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:15:13.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:15:13.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:15:13.483 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:15:13.483 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:15:13.483 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:15:13.483 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:15:13.483 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:15:13.483 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:13.483 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:15:13.483 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.452141/ceph-mon.a.asok 2026-02-17T12:15:13.483 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:15:13.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.452141/ceph-mon.a.asok config get mon_host 2026-02-17T12:15:13.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:179: TEST_scrub_snaps: run_mgr td/osd-scrub-snaps x 2026-02-17T12:15:13.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-snaps 2026-02-17T12:15:13.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:15:13.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:15:13.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:15:13.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-snaps/x 2026-02-17T12:15:13.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:15:13.637 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:15:13.637 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:15:13.637 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:15:13.637 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:15:13.637 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:13.637 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:15:13.637 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:15:13.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:15:13.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-snaps/x '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:15:13.656 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:180: TEST_scrub_snaps: expr 1 - 1 2026-02-17T12:15:13.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:180: TEST_scrub_snaps: seq 0 0 2026-02-17T12:15:13.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:180: TEST_scrub_snaps: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:15:13.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:182: TEST_scrub_snaps: run_osd td/osd-scrub-snaps 0 2026-02-17T12:15:13.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-snaps 2026-02-17T12:15:13.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:15:13.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:15:13.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:15:13.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-17T12:15:13.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-17T12:15:13.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:15:13.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:15:13.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:15:13.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-17T12:15:13.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-17T12:15:13.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:15:13.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:15:13.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-17T12:15:13.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:15:13.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=0fc19e86-d38f-4dd2-80c8-12413d47604e 2026-02-17T12:15:13.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 0fc19e86-d38f-4dd2-80c8-12413d47604e' 2026-02-17T12:15:13.661 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 0fc19e86-d38f-4dd2-80c8-12413d47604e 2026-02-17T12:15:13.661 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:15:13.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDRW5RpwU2xJxAAZId7YnKGVkKtSHYYMjIosA== 2026-02-17T12:15:13.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDRW5RpwU2xJxAAZId7YnKGVkKtSHYYMjIosA=="}' 2026-02-17T12:15:13.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 0fc19e86-d38f-4dd2-80c8-12413d47604e -i td/osd-scrub-snaps/0/new.json 2026-02-17T12:15:13.761 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:15:13.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-snaps/0/new.json 2026-02-17T12:15:13.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDRW5RpwU2xJxAAZId7YnKGVkKtSHYYMjIosA== --osd-uuid 0fc19e86-d38f-4dd2-80c8-12413d47604e 2026-02-17T12:15:13.783 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:15:13.772+0000 7fdd6b4ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:13.784 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:15:13.776+0000 7fdd6b4ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:13.785 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:15:13.776+0000 7fdd6b4ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:13.785 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:15:13.776+0000 7fdd6b4ba8c0 -1 bdev(0x5567d2b81c00 td/osd-scrub-snaps/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:15:13.785 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:15:13.776+0000 7fdd6b4ba8c0 -1 bluestore(td/osd-scrub-snaps/0) _read_fsid unparsable uuid 2026-02-17T12:15:16.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-snaps/0/keyring 2026-02-17T12:15:16.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:15:16.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:15:16.109 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:15:16.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-snaps/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:15:16.300 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:15:16.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:15:16.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:15:16.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:15:16.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:15:16.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:15:16.321 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:15:16.312+0000 7efc39e208c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:16.322 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:15:16.312+0000 7efc39e208c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:16.323 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:15:16.316+0000 7efc39e208c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:15:16.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:15:16.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:15:16.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:15:16.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:15:16.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:15:16.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:16.450 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:15:16.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:15:16.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:16.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:15:16.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:17.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:17.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:17.603 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:15:17.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:15:17.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:17.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:15:17.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:17.849 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:15:17.840+0000 7efc39e208c0 -1 Falling back to public interface 2026-02-17T12:15:18.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:18.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:18.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:15:18.758 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:15:18.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:18.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:15:18.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:19.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:19.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:19.914 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:15:19.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:15:19.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:19.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:15:20.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:20.864 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:15:20.856+0000 7efc39e208c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:15:21.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:21.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:21.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:15:21.067 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:15:21.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:21.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:15:21.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:15:22.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:15:22.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:15:22.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:15:22.240 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:15:22.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:15:22.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:15:22.391 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/667628695,v1:127.0.0.1:6803/667628695] [v2:127.0.0.1:6804/667628695,v1:127.0.0.1:6805/667628695] exists,up 0fc19e86-d38f-4dd2-80c8-12413d47604e 2026-02-17T12:15:22.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:15:22.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:15:22.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:15:22.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:186: TEST_scrub_snaps: ceph osd set noscrub 2026-02-17T12:15:22.591 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:15:22.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:187: TEST_scrub_snaps: ceph osd set nodeep-scrub 2026-02-17T12:15:22.796 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:15:22.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:190: TEST_scrub_snaps: create_pool test 1 1 2026-02-17T12:15:22.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T12:15:22.999 INFO:tasks.workunit.client.0.trial128.stderr:pool 'test' created 2026-02-17T12:15:23.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:15:24.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:191: TEST_scrub_snaps: wait_for_clean 2026-02-17T12:15:24.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:15:24.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:15:24.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:15:24.012 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:15:24.013 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:15:24.013 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:15:24.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:15:24.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:15:24.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:15:24.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:15:24.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:15:24.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:15:24.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:15:24.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:15:24.111 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:15:24.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T12:15:24.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:15:24.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:15:24.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:15:24.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836482 2026-02-17T12:15:24.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836482 2026-02-17T12:15:24.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836482' 2026-02-17T12:15:24.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:15:24.338 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836482 2026-02-17T12:15:24.338 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:15:24.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:15:24.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836482 2026-02-17T12:15:24.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:15:24.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836482 2026-02-17T12:15:24.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836482' 2026-02-17T12:15:24.341 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836482 2026-02-17T12:15:24.342 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:15:24.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836481 -lt 21474836482 2026-02-17T12:15:24.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:15:25.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:15:25.502 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:15:25.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836481 -lt 21474836482 2026-02-17T12:15:25.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:15:26.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:15:26.653 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:15:26.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836482 2026-02-17T12:15:26.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:15:26.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:15:26.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:15:26.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:15:26.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:15:26.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:15:26.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:15:26.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:15:26.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:15:26.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:15:26.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:15:27.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:15:27.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:15:27.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:15:27.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:15:27.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:15:27.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:15:27.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:15:27.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:192: TEST_scrub_snaps: ceph osd dump 2026-02-17T12:15:27.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:192: TEST_scrub_snaps: grep '^pool.*['\'']test['\'']' 2026-02-17T12:15:27.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:192: TEST_scrub_snaps: awk '{ print $2 }' 2026-02-17T12:15:27.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:192: TEST_scrub_snaps: poolid=1 2026-02-17T12:15:27.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:194: TEST_scrub_snaps: dd if=/dev/urandom of=testdata.452141 bs=1032 count=1 2026-02-17T12:15:27.488 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records in 2026-02-17T12:15:27.488 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records out 2026-02-17T12:15:27.488 INFO:tasks.workunit.client.0.trial128.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 5.5042e-05 s, 18.7 MB/s 2026-02-17T12:15:27.489 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: seq 1 16 2026-02-17T12:15:27.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T12:15:27.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj1 testdata.452141 2026-02-17T12:15:27.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T12:15:27.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj2 testdata.452141 2026-02-17T12:15:27.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T12:15:27.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj3 testdata.452141 2026-02-17T12:15:27.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T12:15:27.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj4 testdata.452141 2026-02-17T12:15:27.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T12:15:27.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj5 testdata.452141 2026-02-17T12:15:27.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T12:15:27.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj6 testdata.452141 2026-02-17T12:15:27.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T12:15:27.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj7 testdata.452141 2026-02-17T12:15:27.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T12:15:27.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj8 testdata.452141 2026-02-17T12:15:27.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T12:15:27.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj9 testdata.452141 2026-02-17T12:15:27.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T12:15:27.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj10 testdata.452141 2026-02-17T12:15:27.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T12:15:27.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj11 testdata.452141 2026-02-17T12:15:27.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T12:15:27.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj12 testdata.452141 2026-02-17T12:15:27.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T12:15:27.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj13 testdata.452141 2026-02-17T12:15:27.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T12:15:27.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj14 testdata.452141 2026-02-17T12:15:27.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T12:15:27.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj15 testdata.452141 2026-02-17T12:15:27.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:195: TEST_scrub_snaps: for i in `seq 1 $OBJS` 2026-02-17T12:15:27.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:197: TEST_scrub_snaps: rados -p test put obj16 testdata.452141 2026-02-17T12:15:27.832 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:200: TEST_scrub_snaps: get_primary test obj1 2026-02-17T12:15:27.832 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T12:15:27.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T12:15:27.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T12:15:27.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:15:27.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:200: TEST_scrub_snaps: local primary=0 2026-02-17T12:15:27.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:202: TEST_scrub_snaps: create_scenario td/osd-scrub-snaps test testdata.452141 0 2026-02-17T12:15:27.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:45: create_scenario: local dir=td/osd-scrub-snaps 2026-02-17T12:15:27.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:46: create_scenario: local poolname=test 2026-02-17T12:15:27.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:47: create_scenario: local TESTDATA=testdata.452141 2026-02-17T12:15:27.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:48: create_scenario: local osd=0 2026-02-17T12:15:27.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:50: create_scenario: SNAP=1 2026-02-17T12:15:27.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:51: create_scenario: rados -p test mksnap snap1 2026-02-17T12:15:28.062 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap1 2026-02-17T12:15:28.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:52: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=1 2026-02-17T12:15:28.065 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records in 2026-02-17T12:15:28.065 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records out 2026-02-17T12:15:28.065 INFO:tasks.workunit.client.0.trial128.stderr:256 bytes copied, 4.834e-05 s, 5.3 MB/s 2026-02-17T12:15:28.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:53: create_scenario: rados -p test put obj1 testdata.452141 2026-02-17T12:15:28.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:54: create_scenario: rados -p test put obj5 testdata.452141 2026-02-17T12:15:28.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:55: create_scenario: rados -p test put obj3 testdata.452141 2026-02-17T12:15:28.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: seq 6 14 2026-02-17T12:15:28.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:15:28.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj6 testdata.452141 2026-02-17T12:15:28.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:15:28.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj7 testdata.452141 2026-02-17T12:15:28.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:15:28.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj8 testdata.452141 2026-02-17T12:15:28.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:15:28.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj9 testdata.452141 2026-02-17T12:15:28.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:15:28.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj10 testdata.452141 2026-02-17T12:15:28.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:15:28.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj11 testdata.452141 2026-02-17T12:15:28.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:15:28.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj12 testdata.452141 2026-02-17T12:15:28.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:15:28.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj13 testdata.452141 2026-02-17T12:15:28.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:15:28.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj14 testdata.452141 2026-02-17T12:15:28.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:60: create_scenario: SNAP=2 2026-02-17T12:15:28.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:61: create_scenario: rados -p test mksnap snap2 2026-02-17T12:15:28.418 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap2 2026-02-17T12:15:28.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:62: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=2 2026-02-17T12:15:28.422 INFO:tasks.workunit.client.0.trial128.stderr:2+0 records in 2026-02-17T12:15:28.422 INFO:tasks.workunit.client.0.trial128.stderr:2+0 records out 2026-02-17T12:15:28.422 INFO:tasks.workunit.client.0.trial128.stderr:512 bytes copied, 8.9556e-05 s, 5.7 MB/s 2026-02-17T12:15:28.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:63: create_scenario: rados -p test put obj5 testdata.452141 2026-02-17T12:15:28.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:65: create_scenario: SNAP=3 2026-02-17T12:15:28.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:66: create_scenario: rados -p test mksnap snap3 2026-02-17T12:15:28.520 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap3 2026-02-17T12:15:28.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:67: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=3 2026-02-17T12:15:28.523 INFO:tasks.workunit.client.0.trial128.stderr:3+0 records in 2026-02-17T12:15:28.523 INFO:tasks.workunit.client.0.trial128.stderr:3+0 records out 2026-02-17T12:15:28.523 INFO:tasks.workunit.client.0.trial128.stderr:768 bytes copied, 9.4996e-05 s, 8.1 MB/s 2026-02-17T12:15:28.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:68: create_scenario: rados -p test put obj3 testdata.452141 2026-02-17T12:15:28.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:70: create_scenario: SNAP=4 2026-02-17T12:15:28.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:71: create_scenario: rados -p test mksnap snap4 2026-02-17T12:15:28.622 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap4 2026-02-17T12:15:28.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:72: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=4 2026-02-17T12:15:28.625 INFO:tasks.workunit.client.0.trial128.stderr:4+0 records in 2026-02-17T12:15:28.625 INFO:tasks.workunit.client.0.trial128.stderr:4+0 records out 2026-02-17T12:15:28.625 INFO:tasks.workunit.client.0.trial128.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 8.9156e-05 s, 11.5 MB/s 2026-02-17T12:15:28.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:73: create_scenario: rados -p test put obj5 testdata.452141 2026-02-17T12:15:28.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:74: create_scenario: rados -p test put obj2 testdata.452141 2026-02-17T12:15:28.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:76: create_scenario: SNAP=5 2026-02-17T12:15:28.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:77: create_scenario: rados -p test mksnap snap5 2026-02-17T12:15:28.724 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap5 2026-02-17T12:15:28.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:78: create_scenario: SNAP=6 2026-02-17T12:15:28.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:79: create_scenario: rados -p test mksnap snap6 2026-02-17T12:15:28.826 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap6 2026-02-17T12:15:28.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:80: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=6 2026-02-17T12:15:28.828 INFO:tasks.workunit.client.0.trial128.stderr:6+0 records in 2026-02-17T12:15:28.829 INFO:tasks.workunit.client.0.trial128.stderr:6+0 records out 2026-02-17T12:15:28.829 INFO:tasks.workunit.client.0.trial128.stderr:1536 bytes (1.5 kB, 1.5 KiB) copied, 6.3398e-05 s, 24.2 MB/s 2026-02-17T12:15:28.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:81: create_scenario: rados -p test put obj5 testdata.452141 2026-02-17T12:15:28.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:83: create_scenario: SNAP=7 2026-02-17T12:15:28.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:84: create_scenario: rados -p test mksnap snap7 2026-02-17T12:15:28.927 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap7 2026-02-17T12:15:28.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:86: create_scenario: rados -p test rm obj4 2026-02-17T12:15:28.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:87: create_scenario: rados -p test rm obj16 2026-02-17T12:15:28.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:88: create_scenario: rados -p test rm obj2 2026-02-17T12:15:28.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:90: create_scenario: kill_daemons td/osd-scrub-snaps TERM osd 2026-02-17T12:15:28.992 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:15:28.992 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:15:28.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:15:28.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:15:28.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:15:30.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:15:30.302 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj1 2026-02-17T12:15:31.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: JSON='["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:15:31.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:95: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' --force remove 2026-02-17T12:15:31.781 INFO:tasks.workunit.client.0.trial128.stdout:WARNING: only removing head with clones present 2026-02-17T12:15:31.782 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-17T12:15:32.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-17T12:15:32.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: grep '"snapid":2' 2026-02-17T12:15:33.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:15:33.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:98: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T12:15:33.788 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:c52c9666:::obj5:2# 2026-02-17T12:15:34.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-17T12:15:34.326 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: grep '"snapid":1' 2026-02-17T12:15:35.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:15:35.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:101: create_scenario: OBJ5SAVE='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:15:35.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:103: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-17T12:15:36.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:104: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T12:15:36.266 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:15:36.266 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T12:15:36.266 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:36.266 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:15:36.266 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T12:15:36.266 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T12:15:36.266 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T12:15:36.266 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T12:15:36.267 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T12:15:36.267 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T12:15:36.267 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:15:36.267 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T12:15:36.267 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T12:15:36.267 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T12:15:36.267 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T12:15:36.267 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:15:36.267 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:15:36.267 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:36.268 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:15:36.268 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:15:36.268 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:15:36.268 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:36.268 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:15:36.268 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:15:36.268 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:15:36.268 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:36.268 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:15:36.268 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:15:36.269 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:15:36.269 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:36.269 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:15:36.269 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:15:36.269 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:15:36.269 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:36.269 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:15:36.269 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:15:36.269 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:15:36.269 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:36.269 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:15:36.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:105: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T12:15:36.270 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:15:36.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:106: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --rmtype nosnapmap '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T12:15:36.902 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:c52c9666:::obj5:1# 2026-02-17T12:15:37.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:108: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-17T12:15:38.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:109: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T12:15:38.516 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:15:38.516 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T12:15:38.516 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:38.516 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:15:38.516 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T12:15:38.516 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T12:15:38.516 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T12:15:38.516 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T12:15:38.516 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:38.517 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:15:38.518 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:15:38.518 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:15:38.518 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:38.518 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:15:38.518 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:15:38.518 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:15:38.518 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:38.518 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:15:38.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:110: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T12:15:38.518 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:15:38.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:111: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-17T12:15:38.519 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-17T12:15:38.519 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: grep '"snapid":4' 2026-02-17T12:15:39.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:15:39.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:114: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=18 2026-02-17T12:15:39.365 INFO:tasks.workunit.client.0.trial128.stderr:18+0 records in 2026-02-17T12:15:39.365 INFO:tasks.workunit.client.0.trial128.stderr:18+0 records out 2026-02-17T12:15:39.365 INFO:tasks.workunit.client.0.trial128.stderr:4608 bytes (4.6 kB, 4.5 KiB) copied, 0.000183781 s, 25.1 MB/s 2026-02-17T12:15:39.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:115: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.452141 2026-02-17T12:15:40.534 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj3 2026-02-17T12:15:41.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: JSON='["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:15:41.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:118: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=15 2026-02-17T12:15:41.381 INFO:tasks.workunit.client.0.trial128.stderr:15+0 records in 2026-02-17T12:15:41.381 INFO:tasks.workunit.client.0.trial128.stderr:15+0 records out 2026-02-17T12:15:41.381 INFO:tasks.workunit.client.0.trial128.stderr:3840 bytes (3.8 kB, 3.8 KiB) copied, 0.000106107 s, 36.2 MB/s 2026-02-17T12:15:41.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:119: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.452141 2026-02-17T12:15:42.570 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj4 2026-02-17T12:15:42.570 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: grep '"snapid":7' 2026-02-17T12:15:43.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: JSON='["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:15:43.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:122: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T12:15:44.051 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:0ee9ae15:::obj4:7# 2026-02-17T12:15:44.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:125: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-17T12:15:45.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:126: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T12:15:45.665 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T12:15:45.665 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:45.665 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:15:45.665 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T12:15:45.665 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T12:15:45.665 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T12:15:45.665 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T12:15:45.665 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T12:15:45.665 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T12:15:45.666 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:15:45.666 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T12:15:45.666 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T12:15:45.666 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T12:15:45.666 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T12:15:45.666 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:15:45.666 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:45.666 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:15:45.666 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:15:45.666 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:45.666 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:15:45.666 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:15:45.666 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:45.667 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:15:45.667 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:15:45.667 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:45.667 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:15:45.667 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:15:45.667 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:45.667 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:15:45.667 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:15:45.667 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:45.667 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:15:45.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:127: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T12:15:45.668 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:45.668 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:45.668 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:45.668 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:45.668 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:45.669 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:45.669 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:15:45.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj16 2026-02-17T12:15:45.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: grep '"snapid":7' 2026-02-17T12:15:46.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: JSON='["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:15:46.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:129: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --rmtype snapmap '["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T12:15:47.132 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:461f8b5e:::obj16:7# 2026-02-17T12:15:47.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:131: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-17T12:15:48.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:132: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T12:15:48.769 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T12:15:48.769 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:15:48.769 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T12:15:48.770 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T12:15:48.770 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T12:15:48.770 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T12:15:48.770 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T12:15:48.770 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T12:15:48.770 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:15:48.770 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T12:15:48.770 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T12:15:48.770 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T12:15:48.770 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T12:15:48.770 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:15:48.770 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:15:48.771 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:15:48.771 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:15:48.771 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:15:48.771 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:15:48.771 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:15:48.771 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:15:48.771 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:15:48.771 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:15:48.771 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:15:48.771 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:15:48.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:133: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T12:15:48.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:134: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-17T12:15:48.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj2 2026-02-17T12:15:49.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: JSON='["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:15:49.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:137: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' rm-attr snapset 2026-02-17T12:15:50.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: echo '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:15:50.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: sed 's/snapid":1/snapid":7/' 2026-02-17T12:15:50.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:15:50.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:141: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=7 2026-02-17T12:15:50.797 INFO:tasks.workunit.client.0.trial128.stderr:7+0 records in 2026-02-17T12:15:50.797 INFO:tasks.workunit.client.0.trial128.stderr:7+0 records out 2026-02-17T12:15:50.797 INFO:tasks.workunit.client.0.trial128.stderr:1792 bytes (1.8 kB, 1.8 KiB) copied, 6.5402e-05 s, 27.4 MB/s 2026-02-17T12:15:50.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:142: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.452141 2026-02-17T12:15:51.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj6 2026-02-17T12:15:52.298 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:15:52.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: JSON='["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:15:52.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:145: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset 2026-02-17T12:15:54.030 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj7 2026-02-17T12:15:54.341 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:15:54.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: JSON='["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:15:54.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:147: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset corrupt 2026-02-17T12:15:56.046 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj8 2026-02-17T12:15:56.356 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:15:56.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: JSON='["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:15:56.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:149: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset seq 2026-02-17T12:15:58.065 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj9 2026-02-17T12:15:58.383 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:15:58.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: JSON='["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:15:58.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:151: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_size 2026-02-17T12:16:00.078 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj10 2026-02-17T12:16:00.391 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:16:00.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: JSON='["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:16:00.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:153: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_overlap 2026-02-17T12:16:02.118 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj11 2026-02-17T12:16:02.422 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:16:02.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: JSON='["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:16:02.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:155: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clones 2026-02-17T12:16:04.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj12 2026-02-17T12:16:04.440 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:16:04.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: JSON='["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:16:04.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:157: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset head 2026-02-17T12:16:06.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj13 2026-02-17T12:16:06.514 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:16:07.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: JSON='["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:16:07.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:159: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset snaps 2026-02-17T12:16:08.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj14 2026-02-17T12:16:08.528 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:16:09.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: JSON='["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:16:09.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:161: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset size 2026-02-17T12:16:10.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:163: create_scenario: echo garbage 2026-02-17T12:16:10.250 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj15 2026-02-17T12:16:10.554 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:16:11.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: JSON='["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:16:11.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:165: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' set-attr snapset td/osd-scrub-snaps/bad 2026-02-17T12:16:12.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:166: create_scenario: rm -f td/osd-scrub-snaps/bad 2026-02-17T12:16:12.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:167: create_scenario: return 0 2026-02-17T12:16:12.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:204: TEST_scrub_snaps: rm -f testdata.452141 2026-02-17T12:16:12.260 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:206: TEST_scrub_snaps: expr 1 - 1 2026-02-17T12:16:12.261 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:206: TEST_scrub_snaps: seq 0 0 2026-02-17T12:16:12.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:206: TEST_scrub_snaps: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:16:12.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:208: TEST_scrub_snaps: activate_osd td/osd-scrub-snaps 0 2026-02-17T12:16:12.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-snaps 2026-02-17T12:16:12.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:16:12.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:16:12.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:16:12.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-17T12:16:12.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-17T12:16:12.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:16:12.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:16:12.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:16:12.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-17T12:16:12.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-17T12:16:12.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:16:12.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:16:12.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T12:16:12.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:16:12.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:16:12.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:16:12.265 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:16:12.265 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:12.265 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:16:12.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:16:12.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:16:12.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:16:12.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T12:16:12.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T12:16:12.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:16:12.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:16:12.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:16:12.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:16:12.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:16:12.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:16:12.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-17T12:16:12.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:16:12.267 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:16:12.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:16:12.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-snaps/0/whoami 2026-02-17T12:16:12.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:16:12.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:16:12.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:16:12.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:16:12.286 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:12.275+0000 7f4dd340a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:12.288 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:12.279+0000 7f4dd340a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:12.289 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:12.279+0000 7f4dd340a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:12.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:16:12.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:16:12.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:16:12.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:16:12.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:16:12.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:12.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:16:12.418 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:16:12.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:12.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:16:12.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:13.537 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:13.527+0000 7f4dd340a8c0 -1 Falling back to public interface 2026-02-17T12:16:13.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:13.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:13.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:16:13.574 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:16:13.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:13.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:16:13.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:14.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:14.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:14.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:16:14.730 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:16:14.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:14.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:16:14.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:15.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:15.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:15.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:16:15.886 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:16:15.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:15.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:16:16.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:16.505 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:16.495+0000 7f4dd340a8c0 -1 osd.0 17 log_to_monitors true 2026-02-17T12:16:17.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:17.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:17.045 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:16:17.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:16:17.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:17.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:16:17.193 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 20 up_thru 20 down_at 18 last_clean_interval [5,17) [v2:127.0.0.1:6802/2821554861,v1:127.0.0.1:6803/2821554861] [v2:127.0.0.1:6804/2821554861,v1:127.0.0.1:6805/2821554861] exists,up 0fc19e86-d38f-4dd2-80c8-12413d47604e 2026-02-17T12:16:17.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:16:17.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:16:17.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:16:17.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:210: TEST_scrub_snaps: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_max 25 2026-02-17T12:16:17.255 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:16:17.255 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_shallow_scrub_chunk_max = '' (not observed, change may require restart) osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-17T12:16:17.255 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:16:17.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:211: TEST_scrub_snaps: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_min 5 2026-02-17T12:16:17.337 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:16:17.338 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_shallow_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T12:16:17.338 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:16:17.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:212: TEST_scrub_snaps: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_seconds 1 2026-02-17T12:16:17.414 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:16:17.415 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_pg_stat_report_interval_max_seconds = '' (not observed, change may require restart) " 2026-02-17T12:16:17.415 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:16:17.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:213: TEST_scrub_snaps: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_epochs 1 2026-02-17T12:16:17.492 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:16:17.492 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_pg_stat_report_interval_max_epochs = '' (not observed, change may require restart) " 2026-02-17T12:16:17.492 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:16:17.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:216: TEST_scrub_snaps: wait_for_clean 2026-02-17T12:16:17.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:16:17.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:16:17.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:16:17.500 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:16:17.501 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:16:17.501 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:16:17.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:16:17.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:16:17.502 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:16:17.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:16:17.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:16:17.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:16:17.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:16:17.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:16:17.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:16:17.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T12:16:17.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:16:17.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:16:17.767 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:16:17.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=85899345922 2026-02-17T12:16:17.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 85899345922 2026-02-17T12:16:17.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-85899345922' 2026-02-17T12:16:17.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:16:17.837 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-85899345922 2026-02-17T12:16:17.838 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:16:17.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:16:17.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-85899345922 2026-02-17T12:16:17.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:16:17.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=85899345922 2026-02-17T12:16:17.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 85899345922' 2026-02-17T12:16:17.841 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 85899345922 2026-02-17T12:16:17.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:16:18.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 85899345922 -lt 85899345922 2026-02-17T12:16:18.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:16:18.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:16:18.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:16:18.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:16:18.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:16:18.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:16:18.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:16:18.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:16:18.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:16:18.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:16:18.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:16:18.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:16:18.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:16:18.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:16:18.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:16:18.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:16:18.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:16:18.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:16:18.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:218: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_shallow_scrub_chunk_max 2026-02-17T12:16:18.594 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:16:18.594 INFO:tasks.workunit.client.0.trial128.stdout: "osd_shallow_scrub_chunk_max": "25" 2026-02-17T12:16:18.594 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:16:18.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:219: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_shallow_scrub_chunk_min 2026-02-17T12:16:18.673 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:16:18.673 INFO:tasks.workunit.client.0.trial128.stdout: "osd_shallow_scrub_chunk_min": "5" 2026-02-17T12:16:18.673 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:16:18.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:220: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_pg_stat_report_interval_max_seconds 2026-02-17T12:16:18.749 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:16:18.749 INFO:tasks.workunit.client.0.trial128.stdout: "osd_pg_stat_report_interval_max_seconds": "1" 2026-02-17T12:16:18.749 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:16:18.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:221: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_pg_stat_report_interval_max_epochs 2026-02-17T12:16:18.826 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:16:18.827 INFO:tasks.workunit.client.0.trial128.stdout: "osd_pg_stat_report_interval_max_epochs": "1" 2026-02-17T12:16:18.827 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:16:18.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:222: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_scrub_chunk_max 2026-02-17T12:16:18.904 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:16:18.904 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_chunk_max": "15" 2026-02-17T12:16:18.904 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:16:18.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:223: TEST_scrub_snaps: ceph tell 'osd.*' config get osd_scrub_chunk_min 2026-02-17T12:16:18.973 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:16:18.973 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_chunk_min": "5" 2026-02-17T12:16:18.973 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:16:18.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:225: TEST_scrub_snaps: local pgid=1.0 2026-02-17T12:16:18.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:226: TEST_scrub_snaps: pg_scrub 1.0 2026-02-17T12:16:18.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=1.0 2026-02-17T12:16:18.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 1.0 2026-02-17T12:16:18.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=1.0 2026-02-17T12:16:18.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T12:16:18.982 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:16:18.982 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:16:18.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:16:18.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:16:18.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:16:19.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T12:16:19.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T12:16:19.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T12:16:19.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T12:16:19.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:16:19.242 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:16:19.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-17T12:16:19.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:16:19.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:16:19.393 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:16:19.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=85899345923 2026-02-17T12:16:19.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 85899345923 2026-02-17T12:16:19.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-85899345923' 2026-02-17T12:16:19.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:16:19.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-85899345923 2026-02-17T12:16:19.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:16:19.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:16:19.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-85899345923 2026-02-17T12:16:19.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:16:19.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=85899345923 2026-02-17T12:16:19.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 85899345923' 2026-02-17T12:16:19.470 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 85899345923 2026-02-17T12:16:19.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:16:19.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 85899345922 -lt 85899345923 2026-02-17T12:16:19.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:16:20.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:16:20.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:16:20.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 85899345923 -lt 85899345923 2026-02-17T12:16:20.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T12:16:20.776 INFO:tasks.workunit.client.0.trial128.stdout:#---------- 1.0 loop 0 2026-02-17T12:16:20.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T12:16:20.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T12:16:20.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=1.0 2026-02-17T12:16:20.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T12:16:20.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 1.0 query 2026-02-17T12:16:20.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T12:16:20.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T12:16:20.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T12:16:20.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T12:16:20.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T12:16:20.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 1.0 2026-02-17T12:16:20.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:16:20.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:16:20.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:16:20.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:16:20.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-17T12:15:22.990884+0000 2026-02-17T12:16:20.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 1.0 2026-02-17T12:16:21.139 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 1.0 on osd.0 to scrub 2026-02-17T12:16:21.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 1.0 2026-02-17T12:15:22.990884+0000 2026-02-17T12:16:21.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T12:16:21.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:15:22.990884+0000 2026-02-17T12:16:21.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:16:21.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:16:21.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:16:21.149 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:16:21.149 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:16:21.149 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:16:21.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:16:21.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:16:21.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:15:22.990884+0000 '>' 2026-02-17T12:15:22.990884+0000 2026-02-17T12:16:21.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:16:22.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:16:22.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:16:22.303 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:16:22.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:16:22.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:16:22.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:16:22.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:16:22.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:16:21.551213+0000 '>' 2026-02-17T12:15:22.990884+0000 2026-02-17T12:16:22.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:16:22.461 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:230: TEST_scrub_snaps: grep '_scan_snaps start' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:22.461 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:230: TEST_scrub_snaps: wc -l 2026-02-17T12:16:22.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:230: TEST_scrub_snaps: test 2 = 2 2026-02-17T12:16:22.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:232: TEST_scrub_snaps: rados list-inconsistent-pg test 2026-02-17T12:16:22.489 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:234: TEST_scrub_snaps: jq '. | length' td/osd-scrub-snaps/json 2026-02-17T12:16:22.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:234: TEST_scrub_snaps: test 1 = 1 2026-02-17T12:16:22.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:236: TEST_scrub_snaps: jq -r '.[0]' td/osd-scrub-snaps/json 2026-02-17T12:16:22.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:236: TEST_scrub_snaps: test 1.0 = 1.0 2026-02-17T12:16:22.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:238: TEST_scrub_snaps: rados list-inconsistent-obj 1.0 2026-02-17T12:16:22.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:243: TEST_scrub_snaps: jq .inconsistents 2026-02-17T12:16:22.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:243: TEST_scrub_snaps: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-17T12:16:22.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:250: TEST_scrub_snaps: jq .inconsistents td/osd-scrub-snaps/json 2026-02-17T12:16:22.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:250: TEST_scrub_snaps: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-17T12:16:22.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:251: TEST_scrub_snaps: multidiff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-17T12:16:22.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-17T12:16:22.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:253: TEST_scrub_snaps: rados list-inconsistent-snapset 1.0 2026-02-17T12:16:22.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:255: TEST_scrub_snaps: jq .inconsistents 2026-02-17T12:16:22.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:255: TEST_scrub_snaps: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-17T12:16:22.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:634: TEST_scrub_snaps: jq .inconsistents td/osd-scrub-snaps/json 2026-02-17T12:16:22.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:634: TEST_scrub_snaps: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-17T12:16:22.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:635: TEST_scrub_snaps: multidiff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-17T12:16:22.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-17T12:16:22.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:636: TEST_scrub_snaps: test no = yes 2026-02-17T12:16:22.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:641: TEST_scrub_snaps: test '' = yes 2026-02-17T12:16:22.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:646: TEST_scrub_snaps: find td/osd-scrub-snaps 2026-02-17T12:16:22.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:646: TEST_scrub_snaps: grep 'osd[^/]*\.pid' 2026-02-17T12:16:22.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:646: TEST_scrub_snaps: pidfiles=td/osd-scrub-snaps/osd.0.pid 2026-02-17T12:16:22.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:647: TEST_scrub_snaps: pids= 2026-02-17T12:16:22.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:648: TEST_scrub_snaps: for pidfile in ${pidfiles} 2026-02-17T12:16:22.610 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:650: TEST_scrub_snaps: cat td/osd-scrub-snaps/osd.0.pid 2026-02-17T12:16:22.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:650: TEST_scrub_snaps: pids+='467597 ' 2026-02-17T12:16:22.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:653: TEST_scrub_snaps: ERRORS=0 2026-02-17T12:16:22.612 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: seq 1 7 2026-02-17T12:16:22.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-17T12:16:22.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap1 2026-02-17T12:16:22.658 INFO:tasks.workunit.client.0.trial128.stdout:removed pool test snap snap1 2026-02-17T12:16:22.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-17T12:16:22.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap2 2026-02-17T12:16:22.760 INFO:tasks.workunit.client.0.trial128.stdout:removed pool test snap snap2 2026-02-17T12:16:22.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-17T12:16:22.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap3 2026-02-17T12:16:22.861 INFO:tasks.workunit.client.0.trial128.stdout:removed pool test snap snap3 2026-02-17T12:16:22.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-17T12:16:22.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap4 2026-02-17T12:16:22.964 INFO:tasks.workunit.client.0.trial128.stdout:removed pool test snap snap4 2026-02-17T12:16:22.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-17T12:16:22.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap5 2026-02-17T12:16:23.066 INFO:tasks.workunit.client.0.trial128.stdout:removed pool test snap snap5 2026-02-17T12:16:23.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-17T12:16:23.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap6 2026-02-17T12:16:23.168 INFO:tasks.workunit.client.0.trial128.stdout:removed pool test snap snap6 2026-02-17T12:16:23.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:655: TEST_scrub_snaps: for i in `seq 1 7` 2026-02-17T12:16:23.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:657: TEST_scrub_snaps: rados -p test rmsnap snap7 2026-02-17T12:16:23.271 INFO:tasks.workunit.client.0.trial128.stdout:removed pool test snap snap7 2026-02-17T12:16:23.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:659: TEST_scrub_snaps: sleep 5 2026-02-17T12:16:28.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:660: TEST_scrub_snaps: local -i loop=0 2026-02-17T12:16:28.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:661: TEST_scrub_snaps: ceph pg dump pgs 2026-02-17T12:16:28.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:661: TEST_scrub_snaps: grep -q snaptrim 2026-02-17T12:16:28.412 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:16:28.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:663: TEST_scrub_snaps: ceph pg dump pgs 2026-02-17T12:16:28.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:663: TEST_scrub_snaps: grep -q snaptrim_error 2026-02-17T12:16:28.572 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:16:28.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:665: TEST_scrub_snaps: break 2026-02-17T12:16:28.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:674: TEST_scrub_snaps: ceph pg dump pgs 2026-02-17T12:16:28.734 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T12:16:28.734 INFO:tasks.workunit.client.0.trial128.stdout:1.0 35 0 0 0 0 23416 0 0 60 0 60 active+clean+inconsistent+snaptrim_error 2026-02-17T12:16:23.264867+0000 23'60 29:140 [0] 0 [0] 0 17'56 2026-02-17T12:16:21.551213+0000 0'0 2026-02-17T12:15:22.990884+0000 7 0 periodic scrub scheduled @ 2026-02-18T22:30:18.154263+0000 33 0 2026-02-17T12:16:28.734 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T12:16:28.734 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T12:16:28.734 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:16:28.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:676: TEST_scrub_snaps: for pid in $pids 2026-02-17T12:16:28.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:678: TEST_scrub_snaps: kill -0 467597 2026-02-17T12:16:28.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:685: TEST_scrub_snaps: kill_daemons td/osd-scrub-snaps 2026-02-17T12:16:28.743 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:16:28.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:16:28.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:16:28.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:16:28.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:16:37.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:16:37.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:687: TEST_scrub_snaps: declare -a err_strings 2026-02-17T12:16:37.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:688: TEST_scrub_snaps: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj10:.* : is missing in clone_overlap' 2026-02-17T12:16:37.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:689: TEST_scrub_snaps: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:7 : no '\''_'\'' attr' 2026-02-17T12:16:37.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:690: TEST_scrub_snaps: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:7 : is an unexpected clone' 2026-02-17T12:16:37.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:691: TEST_scrub_snaps: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:4 : on disk size [(]4608[)] does not match object info size [(]512[)] adjusted for ondisk to [(]512[)]' 2026-02-17T12:16:37.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:692: TEST_scrub_snaps: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj5:head : expected clone .*:::obj5:2' 2026-02-17T12:16:37.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:693: TEST_scrub_snaps: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj5:head : expected clone .*:::obj5:1' 2026-02-17T12:16:37.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:694: TEST_scrub_snaps: err_strings[6]='log_channel[(]cluster[)] log [[]INF[]] : scrub [0-9]*[.]0 .*:::obj5:head : 2 missing clone[(]s[)]' 2026-02-17T12:16:37.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:695: TEST_scrub_snaps: err_strings[7]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj8:head : snaps.seq not set' 2026-02-17T12:16:37.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:696: TEST_scrub_snaps: err_strings[8]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj7:1 : is an unexpected clone' 2026-02-17T12:16:37.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:697: TEST_scrub_snaps: err_strings[9]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj3:head : on disk size [(]3840[)] does not match object info size [(]768[)] adjusted for ondisk to [(]768[)]' 2026-02-17T12:16:37.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:698: TEST_scrub_snaps: err_strings[10]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj6:1 : is an unexpected clone' 2026-02-17T12:16:37.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:699: TEST_scrub_snaps: err_strings[11]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:head : no '\''snapset'\'' attr' 2026-02-17T12:16:37.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:700: TEST_scrub_snaps: err_strings[12]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:7 : clone ignored due to missing snapset' 2026-02-17T12:16:37.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:701: TEST_scrub_snaps: err_strings[13]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:4 : clone ignored due to missing snapset' 2026-02-17T12:16:37.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:702: TEST_scrub_snaps: err_strings[14]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj4:head : expected clone .*:::obj4:7' 2026-02-17T12:16:37.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:703: TEST_scrub_snaps: err_strings[15]='log_channel[(]cluster[)] log [[]INF[]] : scrub [0-9]*[.]0 .*:::obj4:head : 1 missing clone[(]s[)]' 2026-02-17T12:16:37.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:704: TEST_scrub_snaps: err_strings[16]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj1:1 : is an unexpected clone' 2026-02-17T12:16:37.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:705: TEST_scrub_snaps: err_strings[17]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj9:1 : is missing in clone_size' 2026-02-17T12:16:37.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:706: TEST_scrub_snaps: err_strings[18]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj11:1 : is an unexpected clone' 2026-02-17T12:16:37.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:707: TEST_scrub_snaps: err_strings[19]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj14:1 : size 1032 != clone_size 1033' 2026-02-17T12:16:37.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:708: TEST_scrub_snaps: err_strings[20]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 20 errors' 2026-02-17T12:16:37.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:709: TEST_scrub_snaps: err_strings[21]='log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj15:head : can'\''t decode '\''snapset'\'' attr ' 2026-02-17T12:16:37.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:710: TEST_scrub_snaps: err_strings[22]='log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' 2026-02-17T12:16:37.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj10:.* : is missing in clone_overlap' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:7 : no '\''_'\'' attr' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:7 : is an unexpected clone' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*::obj5:4 : on disk size [(]4608[)] does not match object info size [(]512[)] adjusted for ondisk to [(]512[)]' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj5:head : expected clone .*:::obj5:2' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj5:head : expected clone .*:::obj5:1' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]INF[]] : scrub [0-9]*[.]0 .*:::obj5:head : 2 missing clone[(]s[)]' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj8:head : snaps.seq not set' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj7:1 : is an unexpected clone' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj3:head : on disk size [(]3840[)] does not match object info size [(]768[)] adjusted for ondisk to [(]768[)]' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj6:1 : is an unexpected clone' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:head : no '\''snapset'\'' attr' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:7 : clone ignored due to missing snapset' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj2:4 : clone ignored due to missing snapset' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj4:head : expected clone .*:::obj4:7' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]INF[]] : scrub [0-9]*[.]0 .*:::obj4:head : 1 missing clone[(]s[)]' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj1:1 : is an unexpected clone' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj9:1 : is missing in clone_size' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj11:1 : is an unexpected clone' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj14:1 : size 1032 != clone_size 1033' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 20 errors' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : scrub [0-9]*[.]0 .*:::obj15:head : can'\''t decode '\''snapset'\'' attr ' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: for err_string in "${err_strings[@]}" 2026-02-17T12:16:37.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:714: TEST_scrub_snaps: grep 'log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:16:37.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:721: TEST_scrub_snaps: '[' 0 '!=' 0 ']' 2026-02-17T12:16:37.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:727: TEST_scrub_snaps: echo 'TEST PASSED' 2026-02-17T12:16:37.114 INFO:tasks.workunit.client.0.trial128.stdout:TEST PASSED 2026-02-17T12:16:37.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:728: TEST_scrub_snaps: return 0 2026-02-17T12:16:37.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:40: run: teardown td/osd-scrub-snaps 2026-02-17T12:16:37.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-snaps 2026-02-17T12:16:37.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:16:37.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-17T12:16:37.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:16:37.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:16:37.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:16:37.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:16:37.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:16:37.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:16:37.120 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:16:37.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:16:37.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:16:37.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:16:37.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:16:37.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:16:37.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:16:37.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:16:37.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:16:37.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:16:37.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:16:37.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:16:37.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-snaps 2026-02-17T12:16:37.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:16:37.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:37.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:16:37.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.452141 2026-02-17T12:16:37.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:16:37.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:16:37.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:37: run: for func in $funcs 2026-02-17T12:16:37.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:38: run: setup td/osd-scrub-snaps 2026-02-17T12:16:37.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-snaps 2026-02-17T12:16:37.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-snaps 2026-02-17T12:16:37.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-snaps 2026-02-17T12:16:37.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:16:37.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-17T12:16:37.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:16:37.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:16:37.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:16:37.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:16:37.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:16:37.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:16:37.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:16:37.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:16:37.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:16:37.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:16:37.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:16:37.144 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:16:37.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:16:37.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:16:37.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:16:37.146 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:16:37.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:16:37.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:16:37.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-snaps 2026-02-17T12:16:37.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:16:37.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:37.150 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:16:37.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.452141 2026-02-17T12:16:37.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:16:37.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:16:37.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-snaps 2026-02-17T12:16:37.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:16:37.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:37.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:16:37.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.452141 2026-02-17T12:16:37.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:16:37.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:16:37.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:16:37.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-snaps 1' TERM HUP INT 2026-02-17T12:16:37.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:39: run: TEST_scrub_snaps_primary td/osd-scrub-snaps 2026-02-17T12:16:37.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1177: TEST_scrub_snaps_primary: local dir=td/osd-scrub-snaps 2026-02-17T12:16:37.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1178: TEST_scrub_snaps_primary: ORIG_ARGS='--fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-17T12:16:37.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1179: TEST_scrub_snaps_primary: CEPH_ARGS+=' --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T12:16:37.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1180: TEST_scrub_snaps_primary: _scrub_snaps_multi td/osd-scrub-snaps primary 2026-02-17T12:16:37.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:732: _scrub_snaps_multi: local dir=td/osd-scrub-snaps 2026-02-17T12:16:37.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:733: _scrub_snaps_multi: local poolname=test 2026-02-17T12:16:37.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:734: _scrub_snaps_multi: local OBJS=16 2026-02-17T12:16:37.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:735: _scrub_snaps_multi: local OSDS=2 2026-02-17T12:16:37.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:736: _scrub_snaps_multi: local which=primary 2026-02-17T12:16:37.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:738: _scrub_snaps_multi: TESTDATA=testdata.452141 2026-02-17T12:16:37.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:740: _scrub_snaps_multi: run_mon td/osd-scrub-snaps a --osd_pool_default_size=2 2026-02-17T12:16:37.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-snaps 2026-02-17T12:16:37.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:16:37.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:16:37.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:16:37.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-snaps/a 2026-02-17T12:16:37.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-snaps/a --run-dir=td/osd-scrub-snaps --osd_pool_default_size=2 2026-02-17T12:16:37.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:16:37.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:16:37.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:16:37.178 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:16:37.178 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:37.179 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:16:37.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:16:37.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-snaps/a '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-snaps/log --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T12:16:37.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:16:37.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:16:37.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:16:37.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:16:37.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:16:37.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:16:37.211 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:16:37.211 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:16:37.211 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:16:37.211 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:16:37.211 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:37.211 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:16:37.211 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.452141/ceph-mon.a.asok 2026-02-17T12:16:37.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:16:37.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.452141/ceph-mon.a.asok config get fsid 2026-02-17T12:16:37.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:16:37.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:16:37.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:16:37.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:16:37.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:16:37.268 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:16:37.268 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:16:37.268 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:16:37.269 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:16:37.270 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:37.270 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:16:37.270 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.452141/ceph-mon.a.asok 2026-02-17T12:16:37.270 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:16:37.270 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.452141/ceph-mon.a.asok config get mon_host 2026-02-17T12:16:37.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:741: _scrub_snaps_multi: run_mgr td/osd-scrub-snaps x 2026-02-17T12:16:37.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-snaps 2026-02-17T12:16:37.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:16:37.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:16:37.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:16:37.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-snaps/x 2026-02-17T12:16:37.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:16:37.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:16:37.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:16:37.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:16:37.419 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:16:37.419 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:37.419 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:16:37.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:16:37.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:16:37.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-snaps/x '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:16:37.441 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: expr 2 - 1 2026-02-17T12:16:37.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: seq 0 1 2026-02-17T12:16:37.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:16:37.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:744: _scrub_snaps_multi: run_osd td/osd-scrub-snaps 0 2026-02-17T12:16:37.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-snaps 2026-02-17T12:16:37.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:16:37.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:16:37.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:16:37.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-17T12:16:37.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T12:16:37.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:16:37.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:16:37.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:16:37.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-17T12:16:37.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-17T12:16:37.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:16:37.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:16:37.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T12:16:37.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:16:37.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:16:37.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:16:37.447 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:16:37.447 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:37.447 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:16:37.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:16:37.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:16:37.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:16:37.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:16:37.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:16:37.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T12:16:37.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T12:16:37.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:16:37.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:16:37.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:16:37.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:16:37.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:16:37.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:16:37.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-17T12:16:37.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:16:37.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=02d39fbd-a4d9-4809-87f7-200e016caee2 2026-02-17T12:16:37.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 02d39fbd-a4d9-4809-87f7-200e016caee2' 2026-02-17T12:16:37.450 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 02d39fbd-a4d9-4809-87f7-200e016caee2 2026-02-17T12:16:37.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:16:37.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAlXJRpiSwVGxAA4jBYkgB7N3ihB7T5JZM5bQ== 2026-02-17T12:16:37.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAlXJRpiSwVGxAA4jBYkgB7N3ihB7T5JZM5bQ=="}' 2026-02-17T12:16:37.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 02d39fbd-a4d9-4809-87f7-200e016caee2 -i td/osd-scrub-snaps/0/new.json 2026-02-17T12:16:37.555 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:16:37.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-snaps/0/new.json 2026-02-17T12:16:37.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAlXJRpiSwVGxAA4jBYkgB7N3ihB7T5JZM5bQ== --osd-uuid 02d39fbd-a4d9-4809-87f7-200e016caee2 2026-02-17T12:16:37.584 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:37.575+0000 7f8ce324d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:37.585 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:37.575+0000 7f8ce324d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:37.586 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:37.575+0000 7f8ce324d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:37.586 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:37.575+0000 7f8ce324d8c0 -1 bdev(0x5651ec193c00 td/osd-scrub-snaps/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:16:37.587 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:37.575+0000 7f8ce324d8c0 -1 bluestore(td/osd-scrub-snaps/0) _read_fsid unparsable uuid 2026-02-17T12:16:40.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-snaps/0/keyring 2026-02-17T12:16:40.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:16:40.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:16:40.367 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:16:40.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-snaps/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:16:40.565 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:16:40.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:16:40.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:16:40.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:16:40.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:16:40.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:16:40.594 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:40.583+0000 7f7ce81c58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:40.596 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:40.587+0000 7f7ce81c58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:40.597 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:40.587+0000 7f7ce81c58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:40.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:16:40.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:16:40.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:16:40.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:16:40.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:16:40.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:40.727 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:16:40.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:16:40.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:40.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:16:40.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:41.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:41.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:41.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:16:41.883 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:16:41.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:41.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:16:42.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:42.097 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:42.087+0000 7f7ce81c58c0 -1 Falling back to public interface 2026-02-17T12:16:43.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:43.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:43.040 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:16:43.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:16:43.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:43.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:16:43.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:44.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:44.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:44.198 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:16:44.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:16:44.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:44.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:16:44.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:45.089 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:45.079+0000 7f7ce81c58c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:16:45.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:45.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:45.351 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:16:45.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:16:45.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:45.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:16:45.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:46.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:46.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:46.505 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:16:46.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:16:46.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:46.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:16:46.659 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/326155562,v1:127.0.0.1:6803/326155562] [v2:127.0.0.1:6804/326155562,v1:127.0.0.1:6805/326155562] exists,up 02d39fbd-a4d9-4809-87f7-200e016caee2 2026-02-17T12:16:46.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:16:46.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:16:46.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:744: _scrub_snaps_multi: run_osd td/osd-scrub-snaps 1 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-snaps 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-snaps/1 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/1' 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/1/journal' 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:16:46.661 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:16:46.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-snaps/1 2026-02-17T12:16:46.663 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:16:46.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=217ad49a-1fed-4adb-9037-bffd0f934c4e 2026-02-17T12:16:46.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 217ad49a-1fed-4adb-9037-bffd0f934c4e' 2026-02-17T12:16:46.664 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 217ad49a-1fed-4adb-9037-bffd0f934c4e 2026-02-17T12:16:46.664 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:16:46.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAuXJRpq3wgKBAAlEQR8IV1D0WQ3DdcA7S0nQ== 2026-02-17T12:16:46.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAuXJRpq3wgKBAAlEQR8IV1D0WQ3DdcA7S0nQ=="}' 2026-02-17T12:16:46.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 217ad49a-1fed-4adb-9037-bffd0f934c4e -i td/osd-scrub-snaps/1/new.json 2026-02-17T12:16:46.831 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:16:46.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-snaps/1/new.json 2026-02-17T12:16:46.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAuXJRpq3wgKBAAlEQR8IV1D0WQ3DdcA7S0nQ== --osd-uuid 217ad49a-1fed-4adb-9037-bffd0f934c4e 2026-02-17T12:16:46.860 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:46.851+0000 7f32e485d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:46.861 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:46.851+0000 7f32e485d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:46.862 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:46.851+0000 7f32e485d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:46.863 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:46.851+0000 7f32e485d8c0 -1 bdev(0x55702a281c00 td/osd-scrub-snaps/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:16:46.863 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:46.851+0000 7f32e485d8c0 -1 bluestore(td/osd-scrub-snaps/1) _read_fsid unparsable uuid 2026-02-17T12:16:49.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-snaps/1/keyring 2026-02-17T12:16:49.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:16:49.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:16:49.147 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:16:49.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-snaps/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:16:49.341 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:16:49.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:16:49.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:16:49.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:16:49.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:16:49.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:16:49.370 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:49.359+0000 7fb720b868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:49.372 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:49.363+0000 7fb720b868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:49.373 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:49.363+0000 7fb720b868c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:16:49.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:16:49.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:16:49.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:16:49.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:16:49.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:16:49.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:49.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:16:49.501 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:16:49.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:49.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:16:49.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:50.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:50.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:50.661 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:16:50.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:16:50.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:50.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:16:50.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:50.893 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:50.883+0000 7fb720b868c0 -1 Falling back to public interface 2026-02-17T12:16:51.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:51.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:51.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:16:51.814 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:16:51.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:51.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:16:51.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:52.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:52.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:52.968 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:16:52.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:16:52.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:52.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:16:53.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:53.929 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:16:53.919+0000 7fb720b868c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:16:54.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:54.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:54.123 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:16:54.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:16:54.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:54.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:16:54.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:16:55.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:16:55.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:16:55.289 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:16:55.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:16:55.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:16:55.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:16:55.447 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/874180765,v1:127.0.0.1:6811/874180765] [v2:127.0.0.1:6812/874180765,v1:127.0.0.1:6813/874180765] exists,up 217ad49a-1fed-4adb-9037-bffd0f934c4e 2026-02-17T12:16:55.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:16:55.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:16:55.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:16:55.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:748: _scrub_snaps_multi: ceph osd set noscrub 2026-02-17T12:16:55.644 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:16:55.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:749: _scrub_snaps_multi: ceph osd set nodeep-scrub 2026-02-17T12:16:55.850 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:16:55.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:752: _scrub_snaps_multi: create_pool test 1 1 2026-02-17T12:16:55.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T12:16:56.053 INFO:tasks.workunit.client.0.trial128.stderr:pool 'test' created 2026-02-17T12:16:56.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:16:57.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:753: _scrub_snaps_multi: wait_for_clean 2026-02-17T12:16:57.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:16:57.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:16:57.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:16:57.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:16:57.067 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:16:57.067 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:16:57.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:16:57.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:16:57.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:16:57.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:16:57.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:16:57.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:16:57.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:16:57.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:16:57.164 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:16:57.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:16:57.327 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:16:57.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:16:57.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:16:57.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:16:57.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T12:16:57.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T12:16:57.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T12:16:57.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:16:57.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:16:57.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705666 2026-02-17T12:16:57.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705666 2026-02-17T12:16:57.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-38654705666' 2026-02-17T12:16:57.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:16:57.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T12:16:57.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:16:57.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:16:57.480 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T12:16:57.480 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:16:57.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T12:16:57.482 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836484 2026-02-17T12:16:57.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T12:16:57.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:16:57.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T12:16:57.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:16:58.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:16:58.647 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:16:58.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T12:16:58.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:16:58.812 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705666 2026-02-17T12:16:58.812 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:16:58.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:16:58.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705666 2026-02-17T12:16:58.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:16:58.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705666 2026-02-17T12:16:58.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705666' 2026-02-17T12:16:58.816 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705666 2026-02-17T12:16:58.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:16:58.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705666 -lt 38654705666 2026-02-17T12:16:58.976 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:16:58.976 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:16:58.976 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:16:59.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:16:59.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:16:59.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:16:59.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:16:59.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:16:59.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:16:59.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:16:59.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:16:59.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:16:59.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:16:59.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:16:59.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:16:59.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:16:59.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:16:59.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:16:59.511 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: ceph osd dump 2026-02-17T12:16:59.511 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: grep '^pool.*['\'']test['\'']' 2026-02-17T12:16:59.512 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: awk '{ print $2 }' 2026-02-17T12:16:59.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: poolid=1 2026-02-17T12:16:59.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:756: _scrub_snaps_multi: dd if=/dev/urandom of=testdata.452141 bs=1032 count=1 2026-02-17T12:16:59.671 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records in 2026-02-17T12:16:59.671 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records out 2026-02-17T12:16:59.671 INFO:tasks.workunit.client.0.trial128.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.5285e-05 s, 29.2 MB/s 2026-02-17T12:16:59.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: seq 1 16 2026-02-17T12:16:59.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:16:59.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj1 testdata.452141 2026-02-17T12:16:59.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:16:59.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj2 testdata.452141 2026-02-17T12:16:59.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:16:59.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj3 testdata.452141 2026-02-17T12:16:59.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:16:59.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj4 testdata.452141 2026-02-17T12:16:59.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:16:59.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj5 testdata.452141 2026-02-17T12:16:59.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:16:59.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj6 testdata.452141 2026-02-17T12:16:59.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:16:59.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj7 testdata.452141 2026-02-17T12:16:59.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:16:59.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj8 testdata.452141 2026-02-17T12:16:59.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:16:59.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj9 testdata.452141 2026-02-17T12:16:59.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:16:59.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj10 testdata.452141 2026-02-17T12:16:59.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:16:59.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj11 testdata.452141 2026-02-17T12:16:59.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:16:59.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj12 testdata.452141 2026-02-17T12:16:59.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:16:59.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj13 testdata.452141 2026-02-17T12:17:00.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:17:00.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj14 testdata.452141 2026-02-17T12:17:00.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:17:00.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj15 testdata.452141 2026-02-17T12:17:00.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:17:00.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj16 testdata.452141 2026-02-17T12:17:00.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:762: _scrub_snaps_multi: get_primary test obj1 2026-02-17T12:17:00.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T12:17:00.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T12:17:00.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T12:17:00.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:17:00.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:762: _scrub_snaps_multi: local primary=1 2026-02-17T12:17:00.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:763: _scrub_snaps_multi: get_not_primary test obj1 2026-02-17T12:17:00.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=test 2026-02-17T12:17:00.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=obj1 2026-02-17T12:17:00.244 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary test obj1 2026-02-17T12:17:00.244 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T12:17:00.244 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T12:17:00.244 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T12:17:00.244 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:17:00.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T12:17:00.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map test obj1 2026-02-17T12:17:00.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T12:17:00.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:763: _scrub_snaps_multi: local replica=0 2026-02-17T12:17:00.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:765: _scrub_snaps_multi: eval create_scenario td/osd-scrub-snaps test testdata.452141 '$primary' 2026-02-17T12:17:00.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:765: _scrub_snaps_multi: create_scenario td/osd-scrub-snaps test testdata.452141 1 2026-02-17T12:17:00.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:45: create_scenario: local dir=td/osd-scrub-snaps 2026-02-17T12:17:00.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:46: create_scenario: local poolname=test 2026-02-17T12:17:00.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:47: create_scenario: local TESTDATA=testdata.452141 2026-02-17T12:17:00.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:48: create_scenario: local osd=1 2026-02-17T12:17:00.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:50: create_scenario: SNAP=1 2026-02-17T12:17:00.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:51: create_scenario: rados -p test mksnap snap1 2026-02-17T12:17:00.632 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap1 2026-02-17T12:17:00.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:52: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=1 2026-02-17T12:17:00.635 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records in 2026-02-17T12:17:00.635 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records out 2026-02-17T12:17:00.635 INFO:tasks.workunit.client.0.trial128.stderr:256 bytes copied, 8.9226e-05 s, 2.9 MB/s 2026-02-17T12:17:00.635 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:53: create_scenario: rados -p test put obj1 testdata.452141 2026-02-17T12:17:00.663 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:54: create_scenario: rados -p test put obj5 testdata.452141 2026-02-17T12:17:00.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:55: create_scenario: rados -p test put obj3 testdata.452141 2026-02-17T12:17:00.715 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: seq 6 14 2026-02-17T12:17:00.716 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:17:00.716 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj6 testdata.452141 2026-02-17T12:17:00.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:17:00.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj7 testdata.452141 2026-02-17T12:17:00.766 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:17:00.766 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj8 testdata.452141 2026-02-17T12:17:00.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:17:00.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj9 testdata.452141 2026-02-17T12:17:00.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:17:00.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj10 testdata.452141 2026-02-17T12:17:00.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:17:00.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj11 testdata.452141 2026-02-17T12:17:00.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:17:00.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj12 testdata.452141 2026-02-17T12:17:00.909 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:17:00.909 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj13 testdata.452141 2026-02-17T12:17:00.934 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:17:00.934 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj14 testdata.452141 2026-02-17T12:17:00.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:60: create_scenario: SNAP=2 2026-02-17T12:17:00.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:61: create_scenario: rados -p test mksnap snap2 2026-02-17T12:17:01.034 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap2 2026-02-17T12:17:01.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:62: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=2 2026-02-17T12:17:01.038 INFO:tasks.workunit.client.0.trial128.stderr:2+0 records in 2026-02-17T12:17:01.038 INFO:tasks.workunit.client.0.trial128.stderr:2+0 records out 2026-02-17T12:17:01.038 INFO:tasks.workunit.client.0.trial128.stderr:512 bytes copied, 8.7593e-05 s, 5.8 MB/s 2026-02-17T12:17:01.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:63: create_scenario: rados -p test put obj5 testdata.452141 2026-02-17T12:17:01.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:65: create_scenario: SNAP=3 2026-02-17T12:17:01.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:66: create_scenario: rados -p test mksnap snap3 2026-02-17T12:17:01.135 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap3 2026-02-17T12:17:01.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:67: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=3 2026-02-17T12:17:01.138 INFO:tasks.workunit.client.0.trial128.stderr:3+0 records in 2026-02-17T12:17:01.138 INFO:tasks.workunit.client.0.trial128.stderr:3+0 records out 2026-02-17T12:17:01.138 INFO:tasks.workunit.client.0.trial128.stderr:768 bytes copied, 9.701e-05 s, 7.9 MB/s 2026-02-17T12:17:01.139 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:68: create_scenario: rados -p test put obj3 testdata.452141 2026-02-17T12:17:01.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:70: create_scenario: SNAP=4 2026-02-17T12:17:01.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:71: create_scenario: rados -p test mksnap snap4 2026-02-17T12:17:01.238 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap4 2026-02-17T12:17:01.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:72: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=4 2026-02-17T12:17:01.242 INFO:tasks.workunit.client.0.trial128.stderr:4+0 records in 2026-02-17T12:17:01.242 INFO:tasks.workunit.client.0.trial128.stderr:4+0 records out 2026-02-17T12:17:01.242 INFO:tasks.workunit.client.0.trial128.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 9.678e-05 s, 10.6 MB/s 2026-02-17T12:17:01.242 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:73: create_scenario: rados -p test put obj5 testdata.452141 2026-02-17T12:17:01.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:74: create_scenario: rados -p test put obj2 testdata.452141 2026-02-17T12:17:01.302 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:76: create_scenario: SNAP=5 2026-02-17T12:17:01.302 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:77: create_scenario: rados -p test mksnap snap5 2026-02-17T12:17:01.341 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap5 2026-02-17T12:17:01.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:78: create_scenario: SNAP=6 2026-02-17T12:17:01.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:79: create_scenario: rados -p test mksnap snap6 2026-02-17T12:17:01.442 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap6 2026-02-17T12:17:01.444 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:80: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=6 2026-02-17T12:17:01.446 INFO:tasks.workunit.client.0.trial128.stderr:6+0 records in 2026-02-17T12:17:01.446 INFO:tasks.workunit.client.0.trial128.stderr:6+0 records out 2026-02-17T12:17:01.446 INFO:tasks.workunit.client.0.trial128.stderr:1536 bytes (1.5 kB, 1.5 KiB) copied, 0.000104975 s, 14.6 MB/s 2026-02-17T12:17:01.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:81: create_scenario: rados -p test put obj5 testdata.452141 2026-02-17T12:17:01.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:83: create_scenario: SNAP=7 2026-02-17T12:17:01.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:84: create_scenario: rados -p test mksnap snap7 2026-02-17T12:17:01.545 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap7 2026-02-17T12:17:01.547 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:86: create_scenario: rados -p test rm obj4 2026-02-17T12:17:01.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:87: create_scenario: rados -p test rm obj16 2026-02-17T12:17:01.596 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:88: create_scenario: rados -p test rm obj2 2026-02-17T12:17:01.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:90: create_scenario: kill_daemons td/osd-scrub-snaps TERM osd 2026-02-17T12:17:01.624 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:17:01.624 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:17:01.625 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:17:01.625 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:17:01.625 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:17:02.934 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:17:02.934 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj1 2026-02-17T12:17:03.784 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: JSON='["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:03.784 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:95: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' --force remove 2026-02-17T12:17:04.437 INFO:tasks.workunit.client.0.trial128.stdout:WARNING: only removing head with clones present 2026-02-17T12:17:04.437 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-17T12:17:04.975 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --op list obj5 2026-02-17T12:17:04.975 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: grep '"snapid":2' 2026-02-17T12:17:05.821 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:05.821 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:98: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T12:17:06.483 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:c52c9666:::obj5:2# 2026-02-17T12:17:07.025 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --op list obj5 2026-02-17T12:17:07.025 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: grep '"snapid":1' 2026-02-17T12:17:07.873 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:07.873 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:101: create_scenario: OBJ5SAVE='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:07.873 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:103: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/1 list 2026-02-17T12:17:08.964 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:104: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T12:17:08.965 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:17:08.966 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T12:17:08.966 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:08.966 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:17:08.966 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T12:17:08.966 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T12:17:08.966 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T12:17:08.966 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T12:17:08.966 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T12:17:08.966 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T12:17:08.966 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:17:08.967 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T12:17:08.967 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T12:17:08.967 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T12:17:08.967 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T12:17:08.967 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:17:08.967 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:17:08.967 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:08.967 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:17:08.967 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:17:08.967 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:17:08.967 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:08.967 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:17:08.968 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:17:08.968 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:17:08.968 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:08.968 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:17:08.968 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:17:08.968 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:17:08.968 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:08.968 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:17:08.968 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:17:08.968 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:17:08.968 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:08.969 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:17:08.969 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:17:08.969 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:17:08.969 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:08.969 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:17:08.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:105: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T12:17:08.970 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:17:08.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:106: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --rmtype nosnapmap '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T12:17:09.596 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:c52c9666:::obj5:1# 2026-02-17T12:17:10.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:108: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/1 list 2026-02-17T12:17:11.223 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:109: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T12:17:11.225 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:17:11.225 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T12:17:11.225 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:11.225 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:17:11.225 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T12:17:11.226 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T12:17:11.226 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T12:17:11.226 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T12:17:11.226 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T12:17:11.226 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T12:17:11.226 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:17:11.226 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T12:17:11.226 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T12:17:11.226 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T12:17:11.226 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T12:17:11.226 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:17:11.227 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:17:11.227 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:11.227 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:17:11.227 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:17:11.227 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:17:11.227 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:11.227 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:17:11.227 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:17:11.227 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:17:11.227 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:11.228 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:17:11.228 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:17:11.228 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:17:11.228 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:11.228 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:17:11.228 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:17:11.228 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:17:11.228 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:11.228 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:17:11.228 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:17:11.229 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:17:11.229 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:11.229 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:17:11.229 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:110: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T12:17:11.230 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:17:11.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:111: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-17T12:17:11.230 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --op list obj5 2026-02-17T12:17:11.231 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: grep '"snapid":4' 2026-02-17T12:17:12.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:12.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:114: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=18 2026-02-17T12:17:12.094 INFO:tasks.workunit.client.0.trial128.stderr:18+0 records in 2026-02-17T12:17:12.094 INFO:tasks.workunit.client.0.trial128.stderr:18+0 records out 2026-02-17T12:17:12.094 INFO:tasks.workunit.client.0.trial128.stderr:4608 bytes (4.6 kB, 4.5 KiB) copied, 0.00016173 s, 28.5 MB/s 2026-02-17T12:17:12.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:115: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.452141 2026-02-17T12:17:13.273 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj3 2026-02-17T12:17:14.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: JSON='["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:14.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:118: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=15 2026-02-17T12:17:14.124 INFO:tasks.workunit.client.0.trial128.stderr:15+0 records in 2026-02-17T12:17:14.125 INFO:tasks.workunit.client.0.trial128.stderr:15+0 records out 2026-02-17T12:17:14.125 INFO:tasks.workunit.client.0.trial128.stderr:3840 bytes (3.8 kB, 3.8 KiB) copied, 0.000115955 s, 33.1 MB/s 2026-02-17T12:17:14.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:119: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.452141 2026-02-17T12:17:15.306 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --op list obj4 2026-02-17T12:17:15.306 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: grep '"snapid":7' 2026-02-17T12:17:16.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: JSON='["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:16.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:122: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T12:17:16.800 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:0ee9ae15:::obj4:7# 2026-02-17T12:17:17.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:125: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/1 list 2026-02-17T12:17:18.427 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:126: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T12:17:18.429 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T12:17:18.429 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:18.429 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:17:18.430 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T12:17:18.430 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T12:17:18.430 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T12:17:18.430 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T12:17:18.430 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T12:17:18.430 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T12:17:18.430 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:17:18.430 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T12:17:18.430 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T12:17:18.430 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T12:17:18.430 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T12:17:18.431 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:17:18.431 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:18.431 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:17:18.431 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:17:18.431 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:18.431 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:17:18.431 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:17:18.431 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:18.431 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:17:18.431 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:17:18.431 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:18.431 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:17:18.432 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:17:18.432 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:18.432 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:17:18.432 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:17:18.432 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:18.432 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:17:18.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:127: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T12:17:18.433 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:18.433 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:18.433 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:18.433 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:18.433 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:18.433 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:18.433 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:17:18.434 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --op list obj16 2026-02-17T12:17:18.434 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: grep '"snapid":7' 2026-02-17T12:17:19.272 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: JSON='["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:19.272 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:129: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --rmtype snapmap '["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T12:17:19.915 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:461f8b5e:::obj16:7# 2026-02-17T12:17:20.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:131: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/1 list 2026-02-17T12:17:21.527 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:132: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T12:17:21.528 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T12:17:21.528 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:17:21.528 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T12:17:21.528 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T12:17:21.528 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T12:17:21.528 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T12:17:21.528 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T12:17:21.528 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T12:17:21.528 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:17:21.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:133: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T12:17:21.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:134: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-17T12:17:21.531 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj2 2026-02-17T12:17:22.376 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: JSON='["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:22.376 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:137: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' rm-attr snapset 2026-02-17T12:17:23.562 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: echo '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:23.562 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: sed 's/snapid":1/snapid":7/' 2026-02-17T12:17:23.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:23.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:141: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=7 2026-02-17T12:17:23.564 INFO:tasks.workunit.client.0.trial128.stderr:7+0 records in 2026-02-17T12:17:23.564 INFO:tasks.workunit.client.0.trial128.stderr:7+0 records out 2026-02-17T12:17:23.565 INFO:tasks.workunit.client.0.trial128.stderr:1792 bytes (1.8 kB, 1.8 KiB) copied, 9.5036e-05 s, 18.9 MB/s 2026-02-17T12:17:23.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:142: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.452141 2026-02-17T12:17:24.738 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj6 2026-02-17T12:17:25.047 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:17:25.583 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: JSON='["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:25.584 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:145: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset 2026-02-17T12:17:26.749 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj7 2026-02-17T12:17:27.058 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:17:27.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: JSON='["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:27.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:147: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset corrupt 2026-02-17T12:17:28.794 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj8 2026-02-17T12:17:29.104 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:17:29.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: JSON='["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:29.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:149: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset seq 2026-02-17T12:17:30.842 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj9 2026-02-17T12:17:31.152 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:17:31.687 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: JSON='["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:31.687 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:151: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_size 2026-02-17T12:17:32.874 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj10 2026-02-17T12:17:33.184 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:17:33.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: JSON='["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:33.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:153: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_overlap 2026-02-17T12:17:34.897 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj11 2026-02-17T12:17:35.215 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:17:35.747 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: JSON='["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:35.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:155: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clones 2026-02-17T12:17:36.922 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj12 2026-02-17T12:17:37.241 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:17:37.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: JSON='["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:37.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:157: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset head 2026-02-17T12:17:38.942 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj13 2026-02-17T12:17:39.250 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:17:39.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: JSON='["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:39.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:159: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset snaps 2026-02-17T12:17:40.982 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj14 2026-02-17T12:17:41.292 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:17:41.828 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: JSON='["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:41.828 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:161: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset size 2026-02-17T12:17:42.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:163: create_scenario: echo garbage 2026-02-17T12:17:42.993 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 --head --op list obj15 2026-02-17T12:17:43.304 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:17:43.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: JSON='["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:17:43.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:165: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/1 '["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' set-attr snapset td/osd-scrub-snaps/bad 2026-02-17T12:17:45.021 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:166: create_scenario: rm -f td/osd-scrub-snaps/bad 2026-02-17T12:17:45.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:167: create_scenario: return 0 2026-02-17T12:17:45.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:767: _scrub_snaps_multi: rm -f testdata.452141 2026-02-17T12:17:45.024 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: expr 2 - 1 2026-02-17T12:17:45.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: seq 0 1 2026-02-17T12:17:45.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:17:45.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:771: _scrub_snaps_multi: activate_osd td/osd-scrub-snaps 0 2026-02-17T12:17:45.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-snaps 2026-02-17T12:17:45.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:17:45.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:17:45.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:17:45.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-17T12:17:45.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T12:17:45.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:17:45.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:17:45.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:17:45.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-17T12:17:45.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-17T12:17:45.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:17:45.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:17:45.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T12:17:45.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:17:45.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:17:45.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:17:45.031 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:17:45.031 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:45.032 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:17:45.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:17:45.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:17:45.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:17:45.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T12:17:45.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T12:17:45.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:17:45.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:17:45.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:17:45.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:17:45.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:17:45.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:17:45.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-17T12:17:45.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:17:45.033 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:17:45.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:17:45.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-snaps/0/whoami 2026-02-17T12:17:45.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:17:45.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:17:45.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:17:45.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:17:45.056 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:17:45.046+0000 7f9034eda8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:45.057 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:17:45.046+0000 7f9034eda8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:45.058 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:17:45.046+0000 7f9034eda8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:45.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:17:45.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:17:45.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:17:45.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:17:45.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:17:45.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:45.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:17:45.187 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:17:45.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:45.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:17:45.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:46.310 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:17:46.298+0000 7f9034eda8c0 -1 Falling back to public interface 2026-02-17T12:17:46.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:46.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:46.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:17:46.346 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:17:46.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:46.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:17:46.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:47.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:47.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:47.501 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:17:47.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:17:47.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:47.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:17:47.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:48.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:48.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:48.663 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:17:48.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:17:48.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:48.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:17:48.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:49.347 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:17:49.334+0000 7f9034eda8c0 -1 osd.0 22 log_to_monitors true 2026-02-17T12:17:49.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:49.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:49.824 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:17:49.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:17:49.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:49.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:17:49.981 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 25 up_thru 25 down_at 23 last_clean_interval [5,22) [v2:127.0.0.1:6802/982195319,v1:127.0.0.1:6803/982195319] [v2:127.0.0.1:6804/982195319,v1:127.0.0.1:6805/982195319] exists,up 02d39fbd-a4d9-4809-87f7-200e016caee2 2026-02-17T12:17:49.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:17:49.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:17:49.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:17:49.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:17:49.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:771: _scrub_snaps_multi: activate_osd td/osd-scrub-snaps 1 2026-02-17T12:17:49.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-snaps 2026-02-17T12:17:49.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:17:49.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:17:49.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:17:49.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-snaps/1 2026-02-17T12:17:49.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T12:17:49.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:17:49.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:17:49.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:17:49.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/1' 2026-02-17T12:17:49.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/1/journal' 2026-02-17T12:17:49.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:17:49.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:17:49.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T12:17:49.987 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:17:49.987 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:17:49.987 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:17:49.987 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:17:49.987 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:17:49.987 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:17:49.987 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:17:49.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:17:49.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:17:49.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T12:17:49.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T12:17:49.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:17:49.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:17:49.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:17:49.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:17:49.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:17:49.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:17:49.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-snaps/1 2026-02-17T12:17:49.988 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:17:49.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:17:49.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:17:49.990 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-snaps/1/whoami 2026-02-17T12:17:49.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:17:49.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:17:49.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:17:49.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:17:50.011 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:17:49.998+0000 7fc2b176c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:50.012 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:17:50.002+0000 7fc2b176c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:50.013 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:17:50.002+0000 7fc2b176c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:17:50.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:17:50.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:17:50.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:17:50.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:17:50.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:17:50.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:50.150 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:17:50.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:17:50.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:50.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:17:50.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:51.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:51.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:51.312 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:17:51.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:17:51.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:51.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:17:51.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:51.541 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:17:51.530+0000 7fc2b176c8c0 -1 Falling back to public interface 2026-02-17T12:17:52.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:52.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:52.476 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:17:52.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:17:52.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:52.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:17:52.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:53.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:53.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:53.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:17:53.639 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:17:53.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:53.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:17:53.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:17:54.564 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:17:54.553+0000 7fc2b176c8c0 -1 osd.1 22 log_to_monitors true 2026-02-17T12:17:54.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:17:54.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:17:54.803 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:17:54.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:17:54.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:17:54.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:17:54.961 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 29 up_thru 29 down_at 23 last_clean_interval [9,22) [v2:127.0.0.1:6810/1508928234,v1:127.0.0.1:6811/1508928234] [v2:127.0.0.1:6812/1508928234,v1:127.0.0.1:6813/1508928234] exists,up 217ad49a-1fed-4adb-9037-bffd0f934c4e 2026-02-17T12:17:54.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:17:54.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:17:54.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:17:54.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:774: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_max 3 2026-02-17T12:17:55.031 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T12:17:55.031 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_shallow_scrub_chunk_max = '' (not observed, change may require restart) osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-17T12:17:55.032 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:17:55.037 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T12:17:55.038 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_shallow_scrub_chunk_max = '' (not observed, change may require restart) osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-17T12:17:55.038 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:17:55.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:775: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_min 3 2026-02-17T12:17:55.127 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T12:17:55.127 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_shallow_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T12:17:55.127 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:17:55.133 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T12:17:55.133 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_shallow_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T12:17:55.133 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:17:55.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:776: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_scrub_chunk_min 3 2026-02-17T12:17:55.215 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T12:17:55.215 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T12:17:55.215 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:17:55.222 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T12:17:55.222 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T12:17:55.222 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:17:55.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:777: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_seconds 1 2026-02-17T12:17:55.306 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T12:17:55.306 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_pg_stat_report_interval_max_seconds = '' (not observed, change may require restart) " 2026-02-17T12:17:55.306 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:17:55.312 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T12:17:55.312 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_pg_stat_report_interval_max_seconds = '' (not observed, change may require restart) " 2026-02-17T12:17:55.312 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:17:55.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:778: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_epochs 1 2026-02-17T12:17:55.387 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T12:17:55.387 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_pg_stat_report_interval_max_epochs = '' (not observed, change may require restart) " 2026-02-17T12:17:55.387 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:17:55.393 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T12:17:55.393 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_pg_stat_report_interval_max_epochs = '' (not observed, change may require restart) " 2026-02-17T12:17:55.393 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:17:55.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:779: _scrub_snaps_multi: wait_for_clean 2026-02-17T12:17:55.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:17:55.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:17:55.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:17:55.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:17:55.402 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:17:55.403 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:17:55.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:17:55.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:17:55.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:17:55.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:17:55.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:17:55.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:17:55.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:17:55.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:17:55.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:17:55.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:17:55.669 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:17:55.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:17:55.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:55.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:17:55.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182403 2026-02-17T12:17:55.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182403 2026-02-17T12:17:55.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182403' 2026-02-17T12:17:55.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:55.746 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:17:55.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051586 2026-02-17T12:17:55.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051586 2026-02-17T12:17:55.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182403 1-124554051586' 2026-02-17T12:17:55.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:17:55.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-107374182403 2026-02-17T12:17:55.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:17:55.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:17:55.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-107374182403 2026-02-17T12:17:55.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:17:55.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182403 2026-02-17T12:17:55.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 107374182403' 2026-02-17T12:17:55.825 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 107374182403 2026-02-17T12:17:55.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:17:55.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182402 -lt 107374182403 2026-02-17T12:17:55.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:17:56.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:17:56.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:17:57.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182402 -lt 107374182403 2026-02-17T12:17:57.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:17:58.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:17:58.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:17:58.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182403 -lt 107374182403 2026-02-17T12:17:58.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:17:58.315 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-124554051586 2026-02-17T12:17:58.315 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:17:58.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:17:58.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-124554051586 2026-02-17T12:17:58.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:17:58.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051586 2026-02-17T12:17:58.318 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 124554051586 2026-02-17T12:17:58.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 124554051586' 2026-02-17T12:17:58.319 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:17:58.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051586 -lt 124554051586 2026-02-17T12:17:58.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:17:58.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:17:58.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:17:58.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:17:58.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:17:58.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:17:58.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:17:58.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:17:58.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:17:58.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:17:58.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:17:58.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:17:58.832 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:17:58.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:17:58.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:17:59.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:17:59.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:17:59.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:17:59.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:781: _scrub_snaps_multi: local pgid=1.0 2026-02-17T12:17:59.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:782: _scrub_snaps_multi: pg_scrub 1.0 2026-02-17T12:17:59.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=1.0 2026-02-17T12:17:59.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 1.0 2026-02-17T12:17:59.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=1.0 2026-02-17T12:17:59.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T12:17:59.026 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:17:59.026 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:17:59.027 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:17:59.027 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:17:59.027 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:17:59.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T12:17:59.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T12:17:59.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T12:17:59.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T12:17:59.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:17:59.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:17:59.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:17:59.444 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:17:59.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:17:59.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:59.445 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:17:59.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182405 2026-02-17T12:17:59.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182405 2026-02-17T12:17:59.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182405' 2026-02-17T12:17:59.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:17:59.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:17:59.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=124554051587 2026-02-17T12:17:59.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 124554051587 2026-02-17T12:17:59.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182405 1-124554051587' 2026-02-17T12:17:59.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:17:59.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-107374182405 2026-02-17T12:17:59.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:17:59.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:17:59.604 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-107374182405 2026-02-17T12:17:59.604 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:17:59.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182405 2026-02-17T12:17:59.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 107374182405' 2026-02-17T12:17:59.606 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 107374182405 2026-02-17T12:17:59.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:17:59.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182403 -lt 107374182405 2026-02-17T12:17:59.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:18:00.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:18:00.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:18:00.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182405 -lt 107374182405 2026-02-17T12:18:00.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:00.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-124554051587 2026-02-17T12:18:00.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:00.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:18:00.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-124554051587 2026-02-17T12:18:00.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:00.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=124554051587 2026-02-17T12:18:00.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 124554051587' 2026-02-17T12:18:00.938 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 124554051587 2026-02-17T12:18:00.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:18:01.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 124554051588 -lt 124554051587 2026-02-17T12:18:01.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T12:18:01.107 INFO:tasks.workunit.client.0.trial128.stdout:#---------- 1.0 loop 0 2026-02-17T12:18:01.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T12:18:01.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T12:18:01.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=1.0 2026-02-17T12:18:01.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T12:18:01.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 1.0 query 2026-02-17T12:18:01.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T12:18:01.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T12:18:01.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T12:18:01.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T12:18:01.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T12:18:01.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 1.0 2026-02-17T12:18:01.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:18:01.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:18:01.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:18:01.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:18:01.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-17T12:16:56.044509+0000 2026-02-17T12:18:01.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 1.0 2026-02-17T12:18:01.487 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 1.0 on osd.1 to scrub 2026-02-17T12:18:01.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 1.0 2026-02-17T12:16:56.044509+0000 2026-02-17T12:18:01.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T12:18:01.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:16:56.044509+0000 2026-02-17T12:18:01.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:18:01.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:18:01.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:18:01.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:18:01.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:18:01.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:18:01.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:18:01.497 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:18:01.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:16:56.044509+0000 '>' 2026-02-17T12:16:56.044509+0000 2026-02-17T12:18:01.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:18:02.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:18:02.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:18:02.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:18:02.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:18:02.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:18:02.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:18:02.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:18:02.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:16:56.044509+0000 '>' 2026-02-17T12:16:56.044509+0000 2026-02-17T12:18:02.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:18:03.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:18:03.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:18:03.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:18:03.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:18:03.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:18:03.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:18:03.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:18:03.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:16:56.044509+0000 '>' 2026-02-17T12:16:56.044509+0000 2026-02-17T12:18:03.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:18:04.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:18:04.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:18:04.967 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:18:04.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:18:04.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:18:04.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:18:04.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:18:05.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:16:56.044509+0000 '>' 2026-02-17T12:16:56.044509+0000 2026-02-17T12:18:05.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:18:06.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:18:06.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:18:06.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:18:06.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:18:06.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:18:06.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:18:06.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:18:06.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:18:01.505363+0000 '>' 2026-02-17T12:16:56.044509+0000 2026-02-17T12:18:06.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:18:06.284 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: grep '_scan_snaps start' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:18:06.284 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: wc -l 2026-02-17T12:18:06.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: test 14 -gt 3 2026-02-17T12:18:06.287 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: grep '_scan_snaps start' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:18:06.287 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: wc -l 2026-02-17T12:18:06.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: test 14 -gt 3 2026-02-17T12:18:06.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:789: _scrub_snaps_multi: rados list-inconsistent-pg test 2026-02-17T12:18:06.314 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:791: _scrub_snaps_multi: jq '. | length' td/osd-scrub-snaps/json 2026-02-17T12:18:06.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:791: _scrub_snaps_multi: test 1 = 1 2026-02-17T12:18:06.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:793: _scrub_snaps_multi: jq -r '.[0]' td/osd-scrub-snaps/json 2026-02-17T12:18:06.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:793: _scrub_snaps_multi: test 1.0 = 1.0 2026-02-17T12:18:06.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:795: _scrub_snaps_multi: rados list-inconsistent-obj 1.0 --format=json-pretty 2026-02-17T12:18:06.358 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:18:06.359 INFO:tasks.workunit.client.0.trial128.stdout: "epoch": 29, 2026-02-17T12:18:06.359 INFO:tasks.workunit.client.0.trial128.stdout: "inconsistents": [ 2026-02-17T12:18:06.359 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.359 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.359 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj5", 2026-02-17T12:18:06.359 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.359 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.359 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:18:06.359 INFO:tasks.workunit.client.0.trial128.stdout: "version": 5 2026-02-17T12:18:06.359 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.360 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.360 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:18:06.360 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:18:06.360 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.360 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:18:06.360 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.360 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj5", 2026-02-17T12:18:06.360 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.360 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": 1, 2026-02-17T12:18:06.360 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1718170787, 2026-02-17T12:18:06.360 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.360 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.361 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.361 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.361 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'19", 2026-02-17T12:18:06.361 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'5", 2026-02-17T12:18:06.361 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4189.0:1", 2026-02-17T12:18:06.361 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 5, 2026-02-17T12:18:06.361 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.361 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:16:59.789431+0000", 2026-02-17T12:18:06.361 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:16:59.789805+0000", 2026-02-17T12:18:06.361 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.361 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.361 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.361 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.362 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.362 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.362 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.362 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x8f4532cc", 2026-02-17T12:18:06.362 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.362 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.362 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.362 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.362 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.362 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.362 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.362 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.363 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.363 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.363 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.363 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.363 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.363 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.363 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032 2026-02-17T12:18:06.363 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.363 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.363 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.363 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.363 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.364 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:18:06.364 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.364 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.364 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.364 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.364 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.364 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.364 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj5", 2026-02-17T12:18:06.364 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.364 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.364 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 2, 2026-02-17T12:18:06.364 INFO:tasks.workunit.client.0.trial128.stdout: "version": 20 2026-02-17T12:18:06.364 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.365 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.365 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:18:06.365 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:18:06.365 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.365 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:18:06.365 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.365 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj5", 2026-02-17T12:18:06.365 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.365 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": 2, 2026-02-17T12:18:06.365 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1718170787, 2026-02-17T12:18:06.365 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.365 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.366 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.366 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.366 INFO:tasks.workunit.client.0.trial128.stdout: "version": "17'41", 2026-02-17T12:18:06.366 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "16'20", 2026-02-17T12:18:06.366 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4237.0:1", 2026-02-17T12:18:06.366 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 20, 2026-02-17T12:18:06.366 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.366 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:17:00.677010+0000", 2026-02-17T12:18:06.366 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:17:00.677378+0000", 2026-02-17T12:18:06.366 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.366 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.366 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.366 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.367 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.367 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.367 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.367 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xf9c4faea", 2026-02-17T12:18:06.367 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.367 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.367 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.367 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.367 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.367 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.367 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.367 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.368 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.368 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.368 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.368 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.368 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.368 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.368 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256 2026-02-17T12:18:06.368 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.368 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.368 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.368 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.368 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.368 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:18:06.369 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.369 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.369 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.369 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.369 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.369 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.369 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj5", 2026-02-17T12:18:06.369 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.369 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.369 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 4, 2026-02-17T12:18:06.369 INFO:tasks.workunit.client.0.trial128.stdout: "version": 42 2026-02-17T12:18:06.369 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.369 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.370 INFO:tasks.workunit.client.0.trial128.stdout: "size_mismatch" 2026-02-17T12:18:06.370 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.370 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:18:06.370 INFO:tasks.workunit.client.0.trial128.stdout: "size_mismatch_info", 2026-02-17T12:18:06.370 INFO:tasks.workunit.client.0.trial128.stdout: "obj_size_info_mismatch" 2026-02-17T12:18:06.370 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.370 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:18:06.370 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.370 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj5", 2026-02-17T12:18:06.370 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.370 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": 4, 2026-02-17T12:18:06.370 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1718170787, 2026-02-17T12:18:06.370 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.371 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.371 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.371 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.371 INFO:tasks.workunit.client.0.trial128.stdout: "version": "19'45", 2026-02-17T12:18:06.371 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "17'42", 2026-02-17T12:18:06.371 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4273.0:1", 2026-02-17T12:18:06.371 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 42, 2026-02-17T12:18:06.371 INFO:tasks.workunit.client.0.trial128.stdout: "size": 512, 2026-02-17T12:18:06.371 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:17:01.056908+0000", 2026-02-17T12:18:06.371 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:17:01.058252+0000", 2026-02-17T12:18:06.371 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.371 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.371 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.372 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.372 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.372 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.372 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.372 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xd7ac3a4f", 2026-02-17T12:18:06.372 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.372 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.372 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.372 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.372 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.372 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.372 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.372 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.373 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.373 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.373 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.373 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.373 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.373 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.373 INFO:tasks.workunit.client.0.trial128.stdout: "size": 512 2026-02-17T12:18:06.373 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.373 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.373 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.373 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.373 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.373 INFO:tasks.workunit.client.0.trial128.stdout: "size_mismatch_info", 2026-02-17T12:18:06.374 INFO:tasks.workunit.client.0.trial128.stdout: "obj_size_info_mismatch" 2026-02-17T12:18:06.374 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.374 INFO:tasks.workunit.client.0.trial128.stdout: "size": 4608, 2026-02-17T12:18:06.374 INFO:tasks.workunit.client.0.trial128.stdout: "object_info": { 2026-02-17T12:18:06.374 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.374 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj5", 2026-02-17T12:18:06.374 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.374 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": 4, 2026-02-17T12:18:06.374 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1718170787, 2026-02-17T12:18:06.374 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.374 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.374 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.374 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.374 INFO:tasks.workunit.client.0.trial128.stdout: "version": "19'45", 2026-02-17T12:18:06.375 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "17'42", 2026-02-17T12:18:06.375 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4273.0:1", 2026-02-17T12:18:06.375 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 42, 2026-02-17T12:18:06.375 INFO:tasks.workunit.client.0.trial128.stdout: "size": 512, 2026-02-17T12:18:06.375 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:17:01.056908+0000", 2026-02-17T12:18:06.375 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:17:01.058252+0000", 2026-02-17T12:18:06.375 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.375 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.375 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.375 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.375 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.375 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.375 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.376 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xd7ac3a4f", 2026-02-17T12:18:06.376 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.376 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.376 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.376 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.376 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.376 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.376 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.376 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.376 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.376 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.376 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.376 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.377 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.377 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.377 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj4", 2026-02-17T12:18:06.377 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.377 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.377 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 7, 2026-02-17T12:18:06.377 INFO:tasks.workunit.client.0.trial128.stdout: "version": 4 2026-02-17T12:18:06.377 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.377 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.377 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:18:06.377 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:18:06.377 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.377 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:18:06.378 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.378 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj4", 2026-02-17T12:18:06.378 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.378 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": 7, 2026-02-17T12:18:06.378 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 2826278768, 2026-02-17T12:18:06.378 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.378 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.378 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.378 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.378 INFO:tasks.workunit.client.0.trial128.stdout: "version": "22'51", 2026-02-17T12:18:06.378 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'4", 2026-02-17T12:18:06.378 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4186.0:1", 2026-02-17T12:18:06.378 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 4, 2026-02-17T12:18:06.379 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.379 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:16:59.762212+0000", 2026-02-17T12:18:06.379 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:16:59.762706+0000", 2026-02-17T12:18:06.379 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.379 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.379 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.379 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.379 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.379 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.379 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.380 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x8f4532cc", 2026-02-17T12:18:06.380 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.380 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.380 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.380 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.380 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.380 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.380 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.380 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.380 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.380 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.381 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.381 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.381 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.381 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.381 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032 2026-02-17T12:18:06.381 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.381 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.381 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.381 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.381 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.381 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:18:06.381 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.382 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.382 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.382 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.382 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.382 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.382 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj5", 2026-02-17T12:18:06.382 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.382 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.382 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 7, 2026-02-17T12:18:06.382 INFO:tasks.workunit.client.0.trial128.stdout: "version": 0 2026-02-17T12:18:06.382 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.382 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.383 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:18:06.383 INFO:tasks.workunit.client.0.trial128.stdout: "missing", 2026-02-17T12:18:06.383 INFO:tasks.workunit.client.0.trial128.stdout: "info_missing" 2026-02-17T12:18:06.383 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.383 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.383 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.383 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.383 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.383 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.383 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:18:06.383 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.383 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.384 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.384 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.384 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.384 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.384 INFO:tasks.workunit.client.0.trial128.stdout: "info_missing" 2026-02-17T12:18:06.384 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.384 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1792 2026-02-17T12:18:06.384 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.384 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.384 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.384 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.384 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.384 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj1", 2026-02-17T12:18:06.385 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.385 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.385 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:18:06.385 INFO:tasks.workunit.client.0.trial128.stdout: "version": 18 2026-02-17T12:18:06.385 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.385 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.385 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:18:06.385 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:18:06.385 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.385 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:18:06.385 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.385 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj1", 2026-02-17T12:18:06.385 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.386 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:18:06.386 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1828249343, 2026-02-17T12:18:06.386 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.386 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.386 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.386 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.386 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'18", 2026-02-17T12:18:06.386 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'1", 2026-02-17T12:18:06.386 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4234.0:1", 2026-02-17T12:18:06.386 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 18, 2026-02-17T12:18:06.386 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.386 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:17:00.653699+0000", 2026-02-17T12:18:06.386 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:17:00.654106+0000", 2026-02-17T12:18:06.387 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.387 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.387 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.387 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.387 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.387 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.387 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.387 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xf9c4faea", 2026-02-17T12:18:06.387 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.388 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.388 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.388 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.388 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.388 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.388 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.388 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.388 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.388 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.388 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.388 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.389 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.389 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.389 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256 2026-02-17T12:18:06.389 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.389 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.389 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.389 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.389 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.389 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:18:06.389 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.389 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.390 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.390 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.390 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.390 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.390 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj10", 2026-02-17T12:18:06.390 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.390 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.390 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:18:06.390 INFO:tasks.workunit.client.0.trial128.stdout: "version": 32 2026-02-17T12:18:06.390 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.390 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.390 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_inconsistency" 2026-02-17T12:18:06.391 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.391 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [], 2026-02-17T12:18:06.391 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:18:06.391 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.391 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj10", 2026-02-17T12:18:06.391 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.391 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:18:06.391 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 718195851, 2026-02-17T12:18:06.391 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.391 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.391 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.391 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.391 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'32", 2026-02-17T12:18:06.391 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'10", 2026-02-17T12:18:06.392 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4255.0:1", 2026-02-17T12:18:06.392 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 32, 2026-02-17T12:18:06.392 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.392 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:17:00.844166+0000", 2026-02-17T12:18:06.392 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:17:00.844582+0000", 2026-02-17T12:18:06.392 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.392 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.392 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.392 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.392 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.392 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.392 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.392 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xf9c4faea", 2026-02-17T12:18:06.392 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.393 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.393 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.393 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.393 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.393 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.393 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.393 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.393 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.393 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.393 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.393 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.393 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.393 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.393 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.393 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.394 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:18:06.394 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:18:06.394 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.394 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:18:06.394 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.394 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:18:06.394 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:18:06.394 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:18:06.394 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.394 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.394 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.394 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.394 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.394 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.394 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.395 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.395 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.395 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.395 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.395 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:18:06.395 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:18:06.395 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.395 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:18:06.395 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.395 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "????", 2026-02-17T12:18:06.395 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:18:06.395 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:18:06.395 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.395 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.396 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.396 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.396 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.396 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.396 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.396 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.396 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.396 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj11", 2026-02-17T12:18:06.396 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.396 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.396 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:18:06.396 INFO:tasks.workunit.client.0.trial128.stdout: "version": 34 2026-02-17T12:18:06.396 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.396 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.396 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_inconsistency" 2026-02-17T12:18:06.397 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.397 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [], 2026-02-17T12:18:06.397 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:18:06.397 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.397 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj11", 2026-02-17T12:18:06.397 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.397 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:18:06.397 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 693400951, 2026-02-17T12:18:06.397 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.397 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.397 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.397 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.397 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'34", 2026-02-17T12:18:06.397 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'11", 2026-02-17T12:18:06.397 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4258.0:1", 2026-02-17T12:18:06.398 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 34, 2026-02-17T12:18:06.398 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.398 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:17:00.872877+0000", 2026-02-17T12:18:06.398 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:17:00.873223+0000", 2026-02-17T12:18:06.398 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.398 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.398 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.398 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.398 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.398 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.398 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.398 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xf9c4faea", 2026-02-17T12:18:06.398 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.398 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.398 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.398 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.399 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.399 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.399 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.399 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.399 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.399 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.399 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.399 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.399 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.399 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.399 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.399 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.399 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:18:06.399 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:18:06.399 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.400 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:18:06.400 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.400 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:18:06.400 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:18:06.400 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:18:06.400 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.400 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.400 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.400 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.400 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.400 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.400 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.400 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.400 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.400 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.400 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [] 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj13", 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: "version": 38 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_inconsistency" 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.401 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [], 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj13", 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 2087409765, 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'38", 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'13", 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4264.0:1", 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 38, 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:17:00.925196+0000", 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:17:00.925558+0000", 2026-02-17T12:18:06.402 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xf9c4faea", 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.403 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.404 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]" 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.405 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj14", 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "version": 40 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_inconsistency" 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [], 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj14", 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 2484217095, 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.406 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'40", 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'14", 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4267.0:1", 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 40, 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:17:00.954646+0000", 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:17:00.955026+0000", 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xf9c4faea", 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.407 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.408 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1033, 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.409 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj15", 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "version": 15 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_corrupted" 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj15", 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 612772309, 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.410 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "version": "15'15", 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "0'0", 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4219.0:1", 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 15, 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:17:00.051718+0000", 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:17:00.052046+0000", 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x8f4532cc", 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.411 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 0, 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [] 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.412 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_corrupted" 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": "Z2FyYmFnZQo=" 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj2", 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: "version": 56 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_missing" 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.413 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj2", 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1058988552, 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "version": "22'56", 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "19'48", 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4309.0:1", 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 56, 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "size": 0, 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:17:01.613986+0000", 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:17:01.614327+0000", 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "whiteout", 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "dirty" 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.414 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xffffffff", 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "size": 0, 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 7, 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 4, 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:18:06.415 INFO:tasks.workunit.client.0.trial128.stdout: 4, 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: 3, 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: 2, 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 7, 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1024, 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: 7, 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: 6, 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: 5 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.416 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_missing" 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "size": 0 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj3", 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "version": 44 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "size_mismatch" 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "size_mismatch_info", 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "obj_size_info_mismatch" 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.417 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj3", 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1643547569, 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "version": "18'44", 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "16'22", 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4279.0:1", 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 44, 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "size": 768, 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:17:01.158880+0000", 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:17:01.159222+0000", 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xb97ce941", 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.418 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "size": 768 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "size_mismatch_info", 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "obj_size_info_mismatch" 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "size": 3840, 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "object_info": { 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj3", 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.419 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1643547569, 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "version": "18'44", 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "16'22", 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4279.0:1", 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 44, 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "size": 768, 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:17:01.158880+0000", 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:17:01.159222+0000", 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xb97ce941", 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.420 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj6", 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "version": 24 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_inconsistency" 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [], 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj6", 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 2202164420, 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.421 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'24", 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'6", 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4243.0:1", 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 24, 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:17:00.730550+0000", 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:17:00.731003+0000", 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xf9c4faea", 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.422 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [] 2026-02-17T12:18:06.423 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj7", 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "version": 26 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_inconsistency" 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [], 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj7", 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1552453721, 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'26", 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'7", 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4246.0:1", 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 26, 2026-02-17T12:18:06.424 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:17:00.757075+0000", 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:17:00.757502+0000", 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xf9c4faea", 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:18:06.425 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 0, 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [] 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj8", 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "version": 28 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.426 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_inconsistency" 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [], 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj8", 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 2381834917, 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'28", 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'8", 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4249.0:1", 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 28, 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:17:00.785361+0000", 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:17:00.785783+0000", 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xf9c4faea", 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.427 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.428 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 0, 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj9", 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "version": 30 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_inconsistency" 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [], 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj9", 2026-02-17T12:18:06.429 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 3833113727, 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'30", 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'9", 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4252.0:1", 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 30, 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:17:00.814825+0000", 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:17:00.815130+0000", 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xf9c4faea", 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:18:06.430 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "size": "????", 2026-02-17T12:18:06.431 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:18:06.432 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:18:06.432 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:18:06.432 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.432 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.432 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.432 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.432 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.432 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.432 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:18:06.432 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:18:06.432 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:797: _scrub_snaps_multi: rados list-inconsistent-snapset 1.0 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:800: _scrub_snaps_multi: '[' primary = replica ']' 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:811: _scrub_snaps_multi: scruberrors=30 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:812: _scrub_snaps_multi: jq .inconsistents 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:812: _scrub_snaps_multi: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1066: _scrub_snaps_multi: jq .inconsistents td/osd-scrub-snaps/json 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1066: _scrub_snaps_multi: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1067: _scrub_snaps_multi: multidiff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1068: _scrub_snaps_multi: test no = yes 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1073: _scrub_snaps_multi: test '' = yes 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: find td/osd-scrub-snaps 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: grep 'osd[^/]*\.pid' 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: pidfiles='td/osd-scrub-snaps/osd.1.pid 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr:td/osd-scrub-snaps/osd.0.pid' 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1079: _scrub_snaps_multi: pids= 2026-02-17T12:18:06.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1080: _scrub_snaps_multi: for pidfile in ${pidfiles} 2026-02-17T12:18:06.434 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: cat td/osd-scrub-snaps/osd.1.pid 2026-02-17T12:18:06.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: pids+='486870 ' 2026-02-17T12:18:06.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1080: _scrub_snaps_multi: for pidfile in ${pidfiles} 2026-02-17T12:18:06.434 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: cat td/osd-scrub-snaps/osd.0.pid 2026-02-17T12:18:06.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: pids+='486257 ' 2026-02-17T12:18:06.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1085: _scrub_snaps_multi: ERRORS=0 2026-02-17T12:18:06.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1089: _scrub_snaps_multi: '[' primary = primary ']' 2026-02-17T12:18:06.434 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: seq 1 7 2026-02-17T12:18:06.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-17T12:18:06.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap1 2026-02-17T12:18:06.494 INFO:tasks.workunit.client.0.trial128.stdout:removed pool test snap snap1 2026-02-17T12:18:06.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-17T12:18:06.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap2 2026-02-17T12:18:06.602 INFO:tasks.workunit.client.0.trial128.stdout:removed pool test snap snap2 2026-02-17T12:18:06.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-17T12:18:06.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap3 2026-02-17T12:18:06.704 INFO:tasks.workunit.client.0.trial128.stdout:removed pool test snap snap3 2026-02-17T12:18:06.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-17T12:18:06.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap4 2026-02-17T12:18:06.806 INFO:tasks.workunit.client.0.trial128.stdout:removed pool test snap snap4 2026-02-17T12:18:06.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-17T12:18:06.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap5 2026-02-17T12:18:06.907 INFO:tasks.workunit.client.0.trial128.stdout:removed pool test snap snap5 2026-02-17T12:18:06.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-17T12:18:06.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap6 2026-02-17T12:18:07.010 INFO:tasks.workunit.client.0.trial128.stdout:removed pool test snap snap6 2026-02-17T12:18:07.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1091: _scrub_snaps_multi: for i in `seq 1 7` 2026-02-17T12:18:07.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1093: _scrub_snaps_multi: rados -p test rmsnap snap7 2026-02-17T12:18:07.112 INFO:tasks.workunit.client.0.trial128.stdout:removed pool test snap snap7 2026-02-17T12:18:07.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1095: _scrub_snaps_multi: sleep 5 2026-02-17T12:18:12.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1096: _scrub_snaps_multi: local -i loop=0 2026-02-17T12:18:12.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1097: _scrub_snaps_multi: ceph pg dump pgs 2026-02-17T12:18:12.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1097: _scrub_snaps_multi: grep -q snaptrim 2026-02-17T12:18:12.263 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:18:12.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1099: _scrub_snaps_multi: ceph pg dump pgs 2026-02-17T12:18:12.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1099: _scrub_snaps_multi: grep -q snaptrim_error 2026-02-17T12:18:12.439 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:18:12.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1101: _scrub_snaps_multi: break 2026-02-17T12:18:12.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1111: _scrub_snaps_multi: ceph pg dump pgs 2026-02-17T12:18:12.599 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T12:18:12.599 INFO:tasks.workunit.client.0.trial128.stdout:1.0 36 0 0 0 0 24448 0 0 56 0 56 active+clean+inconsistent+snaptrim_error 2026-02-17T12:18:07.107271+0000 22'56 37:131 [1,0] 1 [1,0] 1 22'56 2026-02-17T12:18:01.505363+0000 0'0 2026-02-17T12:16:56.044509+0000 7 1 periodic scrub scheduled @ 2026-02-18T20:34:32.088873+0000 33 0 2026-02-17T12:18:12.599 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T12:18:12.599 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T12:18:12.599 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:18:12.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1113: _scrub_snaps_multi: for pid in $pids 2026-02-17T12:18:12.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1115: _scrub_snaps_multi: kill -0 486870 2026-02-17T12:18:12.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1113: _scrub_snaps_multi: for pid in $pids 2026-02-17T12:18:12.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1115: _scrub_snaps_multi: kill -0 486257 2026-02-17T12:18:12.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1122: _scrub_snaps_multi: kill_daemons td/osd-scrub-snaps 2026-02-17T12:18:12.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:18:12.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:18:12.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:18:12.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:18:12.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:18:20.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:18:20.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1124: _scrub_snaps_multi: declare -a err_strings 2026-02-17T12:18:20.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1125: _scrub_snaps_multi: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj4:7 : missing' 2026-02-17T12:18:20.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1126: _scrub_snaps_multi: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj3:head : size 3840 != size 768 from auth oi' 2026-02-17T12:18:20.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1127: _scrub_snaps_multi: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:1 : missing' 2026-02-17T12:18:20.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1128: _scrub_snaps_multi: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:2 : missing' 2026-02-17T12:18:20.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1129: _scrub_snaps_multi: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj5:4 : size 4608 != size 512 from auth oi' 2026-02-17T12:18:20.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1130: _scrub_snaps_multi: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid .*:::obj5:7 : failed to pick suitable object info' 2026-02-17T12:18:20.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1131: _scrub_snaps_multi: err_strings[6]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj1:head : missing' 2026-02-17T12:18:20.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1132: _scrub_snaps_multi: err_strings[7]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 30 errors' 2026-02-17T12:18:20.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T12:18:20.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj4:7 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:18:20.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T12:18:20.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj3:head : size 3840 != size 768 from auth oi' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:18:20.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T12:18:20.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:1 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:18:20.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T12:18:20.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:2 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:18:20.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T12:18:20.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj5:4 : size 4608 != size 512 from auth oi' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:18:20.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T12:18:20.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid .*:::obj5:7 : failed to pick suitable object info' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:18:20.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T12:18:20.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj1:head : missing' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:18:20.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T12:18:20.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 30 errors' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:18:20.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1144: _scrub_snaps_multi: declare -a rep_err_strings 2026-02-17T12:18:20.946 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: eval echo '$primary' 2026-02-17T12:18:20.946 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: echo 1 2026-02-17T12:18:20.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: osd=1 2026-02-17T12:18:20.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1146: _scrub_snaps_multi: rep_err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' 2026-02-17T12:18:20.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1147: _scrub_snaps_multi: for err_string in "${rep_err_strings[@]}" 2026-02-17T12:18:20.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1149: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:18:20.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1156: _scrub_snaps_multi: '[' 0 '!=' 0 ']' 2026-02-17T12:18:20.949 INFO:tasks.workunit.client.0.trial128.stdout:TEST PASSED 2026-02-17T12:18:20.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1162: _scrub_snaps_multi: echo 'TEST PASSED' 2026-02-17T12:18:20.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1163: _scrub_snaps_multi: return 0 2026-02-17T12:18:20.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1181: TEST_scrub_snaps_primary: err=0 2026-02-17T12:18:20.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1182: TEST_scrub_snaps_primary: CEPH_ARGS='--fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-17T12:18:20.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1183: TEST_scrub_snaps_primary: return 0 2026-02-17T12:18:20.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:40: run: teardown td/osd-scrub-snaps 2026-02-17T12:18:20.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-snaps 2026-02-17T12:18:20.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:18:20.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-17T12:18:20.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:18:20.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:18:20.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:18:20.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:18:20.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:18:20.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:18:20.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:18:20.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:18:20.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:18:20.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:18:20.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:18:20.959 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:18:20.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:18:20.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:18:20.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:18:20.961 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:18:20.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:18:20.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:18:20.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-snaps 2026-02-17T12:18:20.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:18:20.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:18:20.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:18:20.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.452141 2026-02-17T12:18:20.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:18:20.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:18:20.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:37: run: for func in $funcs 2026-02-17T12:18:20.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:38: run: setup td/osd-scrub-snaps 2026-02-17T12:18:20.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-snaps 2026-02-17T12:18:20.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-snaps 2026-02-17T12:18:20.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-snaps 2026-02-17T12:18:20.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:18:20.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-17T12:18:20.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:18:20.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:18:20.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:18:20.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:18:20.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:18:20.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:18:20.980 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:18:20.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:18:20.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:18:20.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:18:20.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:18:20.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:18:20.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:18:20.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:18:20.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:18:20.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:18:20.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:18:20.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:18:20.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-snaps 2026-02-17T12:18:20.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:18:20.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:18:20.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:18:20.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.452141 2026-02-17T12:18:20.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:18:20.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:18:20.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-snaps 2026-02-17T12:18:20.991 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:18:20.992 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:18:20.992 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:18:20.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.452141 2026-02-17T12:18:20.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:18:20.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:18:20.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:18:20.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-snaps 1' TERM HUP INT 2026-02-17T12:18:20.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:39: run: TEST_scrub_snaps_replica td/osd-scrub-snaps 2026-02-17T12:18:20.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1167: TEST_scrub_snaps_replica: local dir=td/osd-scrub-snaps 2026-02-17T12:18:20.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1168: TEST_scrub_snaps_replica: ORIG_ARGS='--fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-17T12:18:20.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1169: TEST_scrub_snaps_replica: CEPH_ARGS+=' --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T12:18:20.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1170: TEST_scrub_snaps_replica: _scrub_snaps_multi td/osd-scrub-snaps replica 2026-02-17T12:18:20.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:732: _scrub_snaps_multi: local dir=td/osd-scrub-snaps 2026-02-17T12:18:20.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:733: _scrub_snaps_multi: local poolname=test 2026-02-17T12:18:20.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:734: _scrub_snaps_multi: local OBJS=16 2026-02-17T12:18:20.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:735: _scrub_snaps_multi: local OSDS=2 2026-02-17T12:18:20.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:736: _scrub_snaps_multi: local which=replica 2026-02-17T12:18:20.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:738: _scrub_snaps_multi: TESTDATA=testdata.452141 2026-02-17T12:18:20.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:740: _scrub_snaps_multi: run_mon td/osd-scrub-snaps a --osd_pool_default_size=2 2026-02-17T12:18:20.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-snaps 2026-02-17T12:18:20.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:18:20.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:18:20.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:18:20.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-snaps/a 2026-02-17T12:18:20.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-snaps/a --run-dir=td/osd-scrub-snaps --osd_pool_default_size=2 2026-02-17T12:18:21.018 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:18:21.018 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:18:21.018 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:18:21.018 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:18:21.018 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:18:21.018 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:18:21.019 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:18:21.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-snaps/a '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-snaps/log --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T12:18:21.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:18:21.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:18:21.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:18:21.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:18:21.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:18:21.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:18:21.051 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:18:21.052 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:18:21.052 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:18:21.052 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:18:21.052 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:18:21.052 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:18:21.052 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.452141/ceph-mon.a.asok 2026-02-17T12:18:21.052 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:18:21.052 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.452141/ceph-mon.a.asok config get fsid 2026-02-17T12:18:21.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:18:21.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:18:21.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:18:21.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:18:21.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:18:21.107 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:18:21.107 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:18:21.108 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:18:21.108 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:18:21.108 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:18:21.108 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:18:21.108 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.452141/ceph-mon.a.asok 2026-02-17T12:18:21.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:18:21.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.452141/ceph-mon.a.asok config get mon_host 2026-02-17T12:18:21.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:741: _scrub_snaps_multi: run_mgr td/osd-scrub-snaps x 2026-02-17T12:18:21.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-snaps 2026-02-17T12:18:21.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:18:21.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:18:21.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:18:21.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-snaps/x 2026-02-17T12:18:21.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:18:21.258 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:18:21.258 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:18:21.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:18:21.259 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:18:21.259 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:18:21.259 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:18:21.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:18:21.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:18:21.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-snaps/x '--log-file=td/osd-scrub-snaps/$name.log' '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --run-dir=td/osd-scrub-snaps '--pid-file=td/osd-scrub-snaps/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:18:21.282 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: expr 2 - 1 2026-02-17T12:18:21.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: seq 0 1 2026-02-17T12:18:21.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:18:21.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:744: _scrub_snaps_multi: run_osd td/osd-scrub-snaps 0 2026-02-17T12:18:21.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-snaps 2026-02-17T12:18:21.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:18:21.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:18:21.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:18:21.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-17T12:18:21.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T12:18:21.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:18:21.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:18:21.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:18:21.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-17T12:18:21.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-17T12:18:21.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:18:21.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:18:21.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T12:18:21.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:18:21.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:18:21.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:18:21.288 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:18:21.288 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:18:21.288 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:18:21.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:18:21.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:18:21.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:18:21.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:18:21.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:18:21.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T12:18:21.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T12:18:21.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:18:21.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:18:21.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:18:21.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:18:21.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:18:21.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:18:21.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-17T12:18:21.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:18:21.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ff2b90b2-0844-4470-b067-1d09fd2f612d 2026-02-17T12:18:21.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 ff2b90b2-0844-4470-b067-1d09fd2f612d' 2026-02-17T12:18:21.291 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 ff2b90b2-0844-4470-b067-1d09fd2f612d 2026-02-17T12:18:21.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:18:21.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCNXJRpozGWERAAgd8eWs1+FGebDpFs+VgV+A== 2026-02-17T12:18:21.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCNXJRpozGWERAAgd8eWs1+FGebDpFs+VgV+A=="}' 2026-02-17T12:18:21.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ff2b90b2-0844-4470-b067-1d09fd2f612d -i td/osd-scrub-snaps/0/new.json 2026-02-17T12:18:21.394 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:18:21.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-snaps/0/new.json 2026-02-17T12:18:21.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCNXJRpozGWERAAgd8eWs1+FGebDpFs+VgV+A== --osd-uuid ff2b90b2-0844-4470-b067-1d09fd2f612d 2026-02-17T12:18:21.422 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:21.413+0000 7f435b0cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:21.423 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:21.417+0000 7f435b0cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:21.424 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:21.417+0000 7f435b0cc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:21.425 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:21.417+0000 7f435b0cc8c0 -1 bdev(0x563d145b3c00 td/osd-scrub-snaps/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:18:21.425 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:21.417+0000 7f435b0cc8c0 -1 bluestore(td/osd-scrub-snaps/0) _read_fsid unparsable uuid 2026-02-17T12:18:24.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-snaps/0/keyring 2026-02-17T12:18:24.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:18:24.238 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:18:24.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:18:24.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-snaps/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:18:24.432 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:18:24.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:18:24.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:18:24.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:18:24.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:18:24.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:18:24.462 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:24.453+0000 7fd9219448c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:24.464 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:24.457+0000 7fd9219448c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:24.465 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:24.457+0000 7fd9219448c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:24.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:18:24.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:18:24.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:18:24.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:18:24.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:18:24.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:24.597 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:18:24.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:18:24.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:24.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:18:24.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:25.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:25.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:25.753 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:18:25.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:18:25.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:25.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:18:25.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:25.977 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:25.969+0000 7fd9219448c0 -1 Falling back to public interface 2026-02-17T12:18:26.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:26.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:26.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:18:26.909 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:18:26.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:26.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:18:27.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:28.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:28.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:28.065 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:18:28.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:18:28.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:28.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:18:28.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:28.980 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:28.969+0000 7fd9219448c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:18:29.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:29.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:29.221 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:18:29.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:18:29.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:29.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:18:29.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:30.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:30.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:30.378 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:18:30.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:18:30.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:30.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:18:30.535 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/78713230,v1:127.0.0.1:6803/78713230] [v2:127.0.0.1:6804/78713230,v1:127.0.0.1:6805/78713230] exists,up ff2b90b2-0844-4470-b067-1d09fd2f612d 2026-02-17T12:18:30.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:18:30.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:18:30.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:18:30.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:742: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:18:30.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:744: _scrub_snaps_multi: run_osd td/osd-scrub-snaps 1 2026-02-17T12:18:30.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-snaps 2026-02-17T12:18:30.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:18:30.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:18:30.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:18:30.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-snaps/1 2026-02-17T12:18:30.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T12:18:30.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:18:30.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:18:30.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:18:30.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/1' 2026-02-17T12:18:30.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/1/journal' 2026-02-17T12:18:30.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:18:30.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:18:30.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T12:18:30.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:18:30.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:18:30.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:18:30.540 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:18:30.540 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:18:30.540 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:18:30.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:18:30.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:18:30.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:18:30.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:18:30.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:18:30.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T12:18:30.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T12:18:30.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:18:30.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:18:30.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:18:30.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:18:30.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:18:30.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:18:30.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-snaps/1 2026-02-17T12:18:30.541 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:18:30.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=98ad9e01-ae90-4107-a892-afc1bd515b4c 2026-02-17T12:18:30.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 98ad9e01-ae90-4107-a892-afc1bd515b4c' 2026-02-17T12:18:30.543 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 98ad9e01-ae90-4107-a892-afc1bd515b4c 2026-02-17T12:18:30.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:18:30.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCWXJRppB6eIBAAFqDapDzXWjF6zvnOlJmrig== 2026-02-17T12:18:30.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCWXJRppB6eIBAAFqDapDzXWjF6zvnOlJmrig=="}' 2026-02-17T12:18:30.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 98ad9e01-ae90-4107-a892-afc1bd515b4c -i td/osd-scrub-snaps/1/new.json 2026-02-17T12:18:30.714 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:18:30.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-snaps/1/new.json 2026-02-17T12:18:30.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCWXJRppB6eIBAAFqDapDzXWjF6zvnOlJmrig== --osd-uuid 98ad9e01-ae90-4107-a892-afc1bd515b4c 2026-02-17T12:18:30.744 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:30.737+0000 7f6b3ff6a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:30.746 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:30.737+0000 7f6b3ff6a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:30.747 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:30.737+0000 7f6b3ff6a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:30.747 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:30.737+0000 7f6b3ff6a8c0 -1 bdev(0x560cf9853c00 td/osd-scrub-snaps/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:18:30.747 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:30.737+0000 7f6b3ff6a8c0 -1 bluestore(td/osd-scrub-snaps/1) _read_fsid unparsable uuid 2026-02-17T12:18:33.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-snaps/1/keyring 2026-02-17T12:18:33.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:18:33.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:18:33.063 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:18:33.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-snaps/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:18:33.257 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:18:33.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:18:33.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:18:33.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:18:33.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:18:33.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:18:33.287 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:33.277+0000 7fe0a88ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:33.288 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:33.281+0000 7fe0a88ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:33.289 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:33.281+0000 7fe0a88ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:18:33.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:18:33.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:18:33.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:18:33.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:18:33.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:18:33.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:33.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:18:33.416 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:18:33.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:33.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:18:33.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:34.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:34.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:34.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:18:34.570 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:18:34.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:34.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:18:34.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:34.777 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:34.769+0000 7fe0a88ba8c0 -1 Falling back to public interface 2026-02-17T12:18:35.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:35.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:35.725 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:18:35.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:18:35.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:35.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:18:35.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:36.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:36.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:36.882 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:18:36.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:18:36.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:36.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:18:37.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:37.806 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:18:37.797+0000 7fe0a88ba8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:18:38.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:38.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:38.037 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:18:38.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:18:38.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:38.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:18:38.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:18:39.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:18:39.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:18:39.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:18:39.199 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:18:39.199 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:18:39.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:18:39.352 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/218458683,v1:127.0.0.1:6811/218458683] [v2:127.0.0.1:6812/218458683,v1:127.0.0.1:6813/218458683] exists,up 98ad9e01-ae90-4107-a892-afc1bd515b4c 2026-02-17T12:18:39.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:18:39.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:18:39.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:18:39.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:748: _scrub_snaps_multi: ceph osd set noscrub 2026-02-17T12:18:39.550 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:18:39.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:749: _scrub_snaps_multi: ceph osd set nodeep-scrub 2026-02-17T12:18:39.756 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:18:39.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:752: _scrub_snaps_multi: create_pool test 1 1 2026-02-17T12:18:39.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T12:18:39.959 INFO:tasks.workunit.client.0.trial128.stderr:pool 'test' created 2026-02-17T12:18:39.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:18:40.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:753: _scrub_snaps_multi: wait_for_clean 2026-02-17T12:18:40.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:18:40.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:18:40.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:18:40.972 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:18:40.973 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:18:40.973 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:18:40.974 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:18:40.974 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:18:40.974 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:18:41.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:18:41.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:18:41.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:18:41.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:18:41.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:18:41.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:18:41.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:18:41.233 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:18:41.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:18:41.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:41.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:18:41.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836484 2026-02-17T12:18:41.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836484 2026-02-17T12:18:41.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484' 2026-02-17T12:18:41.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:18:41.311 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:18:41.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672962 2026-02-17T12:18:41.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672962 2026-02-17T12:18:41.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836484 1-42949672962' 2026-02-17T12:18:41.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:41.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836484 2026-02-17T12:18:41.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:41.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:18:41.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836484 2026-02-17T12:18:41.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:41.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836484 2026-02-17T12:18:41.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-17T12:18:41.392 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836484 2026-02-17T12:18:41.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:18:41.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-17T12:18:41.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:18:42.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:18:42.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:18:42.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-17T12:18:42.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:18:42.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672962 2026-02-17T12:18:42.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:18:42.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:18:42.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672962 2026-02-17T12:18:42.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:18:42.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672962 2026-02-17T12:18:42.710 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672962 2026-02-17T12:18:42.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672962' 2026-02-17T12:18:42.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:18:42.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672962 -lt 42949672962 2026-02-17T12:18:42.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:18:42.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:18:42.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:18:43.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:18:43.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:18:43.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:18:43.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:18:43.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:18:43.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:18:43.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:18:43.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:18:43.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:18:43.225 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:18:43.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:18:43.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:18:43.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:18:43.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:18:43.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:18:43.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: ceph osd dump 2026-02-17T12:18:43.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: grep '^pool.*['\'']test['\'']' 2026-02-17T12:18:43.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: awk '{ print $2 }' 2026-02-17T12:18:43.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:754: _scrub_snaps_multi: poolid=1 2026-02-17T12:18:43.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:756: _scrub_snaps_multi: dd if=/dev/urandom of=testdata.452141 bs=1032 count=1 2026-02-17T12:18:43.578 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records in 2026-02-17T12:18:43.578 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records out 2026-02-17T12:18:43.578 INFO:tasks.workunit.client.0.trial128.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.3072e-05 s, 31.2 MB/s 2026-02-17T12:18:43.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: seq 1 16 2026-02-17T12:18:43.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:18:43.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj1 testdata.452141 2026-02-17T12:18:43.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:18:43.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj2 testdata.452141 2026-02-17T12:18:43.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:18:43.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj3 testdata.452141 2026-02-17T12:18:43.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:18:43.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj4 testdata.452141 2026-02-17T12:18:43.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:18:43.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj5 testdata.452141 2026-02-17T12:18:43.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:18:43.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj6 testdata.452141 2026-02-17T12:18:43.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:18:43.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj7 testdata.452141 2026-02-17T12:18:43.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:18:43.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj8 testdata.452141 2026-02-17T12:18:43.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:18:43.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj9 testdata.452141 2026-02-17T12:18:43.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:18:43.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj10 testdata.452141 2026-02-17T12:18:43.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:18:43.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj11 testdata.452141 2026-02-17T12:18:43.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:18:43.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj12 testdata.452141 2026-02-17T12:18:43.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:18:43.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj13 testdata.452141 2026-02-17T12:18:43.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:18:43.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj14 testdata.452141 2026-02-17T12:18:43.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:18:43.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj15 testdata.452141 2026-02-17T12:18:43.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:757: _scrub_snaps_multi: for i in `seq 1 $OBJS` 2026-02-17T12:18:43.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:759: _scrub_snaps_multi: rados -p test put obj16 testdata.452141 2026-02-17T12:18:44.019 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:762: _scrub_snaps_multi: get_primary test obj1 2026-02-17T12:18:44.019 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T12:18:44.019 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T12:18:44.019 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T12:18:44.019 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:18:44.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:762: _scrub_snaps_multi: local primary=1 2026-02-17T12:18:44.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:763: _scrub_snaps_multi: get_not_primary test obj1 2026-02-17T12:18:44.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=test 2026-02-17T12:18:44.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=obj1 2026-02-17T12:18:44.176 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary test obj1 2026-02-17T12:18:44.176 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T12:18:44.176 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T12:18:44.176 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T12:18:44.177 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:18:44.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T12:18:44.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map test obj1 2026-02-17T12:18:44.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T12:18:44.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:763: _scrub_snaps_multi: local replica=0 2026-02-17T12:18:44.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:765: _scrub_snaps_multi: eval create_scenario td/osd-scrub-snaps test testdata.452141 '$replica' 2026-02-17T12:18:44.495 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:765: _scrub_snaps_multi: create_scenario td/osd-scrub-snaps test testdata.452141 0 2026-02-17T12:18:44.495 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:45: create_scenario: local dir=td/osd-scrub-snaps 2026-02-17T12:18:44.495 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:46: create_scenario: local poolname=test 2026-02-17T12:18:44.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:47: create_scenario: local TESTDATA=testdata.452141 2026-02-17T12:18:44.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:48: create_scenario: local osd=0 2026-02-17T12:18:44.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:50: create_scenario: SNAP=1 2026-02-17T12:18:44.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:51: create_scenario: rados -p test mksnap snap1 2026-02-17T12:18:44.518 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap1 2026-02-17T12:18:44.520 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:52: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=1 2026-02-17T12:18:44.521 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records in 2026-02-17T12:18:44.521 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records out 2026-02-17T12:18:44.521 INFO:tasks.workunit.client.0.trial128.stderr:256 bytes copied, 5.0805e-05 s, 5.0 MB/s 2026-02-17T12:18:44.521 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:53: create_scenario: rados -p test put obj1 testdata.452141 2026-02-17T12:18:44.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:54: create_scenario: rados -p test put obj5 testdata.452141 2026-02-17T12:18:44.572 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:55: create_scenario: rados -p test put obj3 testdata.452141 2026-02-17T12:18:44.601 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: seq 6 14 2026-02-17T12:18:44.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:18:44.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj6 testdata.452141 2026-02-17T12:18:44.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:18:44.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj7 testdata.452141 2026-02-17T12:18:44.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:18:44.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj8 testdata.452141 2026-02-17T12:18:44.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:18:44.683 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj9 testdata.452141 2026-02-17T12:18:44.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:18:44.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj10 testdata.452141 2026-02-17T12:18:44.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:18:44.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj11 testdata.452141 2026-02-17T12:18:44.753 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:18:44.753 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj12 testdata.452141 2026-02-17T12:18:44.774 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:18:44.774 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj13 testdata.452141 2026-02-17T12:18:44.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:56: create_scenario: for i in `seq 6 14` 2026-02-17T12:18:44.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:57: create_scenario: rados -p test put obj14 testdata.452141 2026-02-17T12:18:44.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:60: create_scenario: SNAP=2 2026-02-17T12:18:44.831 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:61: create_scenario: rados -p test mksnap snap2 2026-02-17T12:18:44.909 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap2 2026-02-17T12:18:44.911 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:62: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=2 2026-02-17T12:18:44.912 INFO:tasks.workunit.client.0.trial128.stderr:2+0 records in 2026-02-17T12:18:44.912 INFO:tasks.workunit.client.0.trial128.stderr:2+0 records out 2026-02-17T12:18:44.912 INFO:tasks.workunit.client.0.trial128.stderr:512 bytes copied, 8.6131e-05 s, 5.9 MB/s 2026-02-17T12:18:44.912 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:63: create_scenario: rados -p test put obj5 testdata.452141 2026-02-17T12:18:44.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:65: create_scenario: SNAP=3 2026-02-17T12:18:44.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:66: create_scenario: rados -p test mksnap snap3 2026-02-17T12:18:45.010 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap3 2026-02-17T12:18:45.012 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:67: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=3 2026-02-17T12:18:45.014 INFO:tasks.workunit.client.0.trial128.stderr:3+0 records in 2026-02-17T12:18:45.014 INFO:tasks.workunit.client.0.trial128.stderr:3+0 records out 2026-02-17T12:18:45.014 INFO:tasks.workunit.client.0.trial128.stderr:768 bytes copied, 9.0518e-05 s, 8.5 MB/s 2026-02-17T12:18:45.014 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:68: create_scenario: rados -p test put obj3 testdata.452141 2026-02-17T12:18:45.044 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:70: create_scenario: SNAP=4 2026-02-17T12:18:45.045 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:71: create_scenario: rados -p test mksnap snap4 2026-02-17T12:18:45.112 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap4 2026-02-17T12:18:45.114 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:72: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=4 2026-02-17T12:18:45.115 INFO:tasks.workunit.client.0.trial128.stderr:4+0 records in 2026-02-17T12:18:45.115 INFO:tasks.workunit.client.0.trial128.stderr:4+0 records out 2026-02-17T12:18:45.115 INFO:tasks.workunit.client.0.trial128.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 9.6139e-05 s, 10.7 MB/s 2026-02-17T12:18:45.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:73: create_scenario: rados -p test put obj5 testdata.452141 2026-02-17T12:18:45.145 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:74: create_scenario: rados -p test put obj2 testdata.452141 2026-02-17T12:18:45.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:76: create_scenario: SNAP=5 2026-02-17T12:18:45.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:77: create_scenario: rados -p test mksnap snap5 2026-02-17T12:18:45.215 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap5 2026-02-17T12:18:45.216 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:78: create_scenario: SNAP=6 2026-02-17T12:18:45.217 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:79: create_scenario: rados -p test mksnap snap6 2026-02-17T12:18:45.316 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap6 2026-02-17T12:18:45.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:80: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=6 2026-02-17T12:18:45.319 INFO:tasks.workunit.client.0.trial128.stderr:6+0 records in 2026-02-17T12:18:45.319 INFO:tasks.workunit.client.0.trial128.stderr:6+0 records out 2026-02-17T12:18:45.319 INFO:tasks.workunit.client.0.trial128.stderr:1536 bytes (1.5 kB, 1.5 KiB) copied, 0.000102671 s, 15.0 MB/s 2026-02-17T12:18:45.319 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:81: create_scenario: rados -p test put obj5 testdata.452141 2026-02-17T12:18:45.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:83: create_scenario: SNAP=7 2026-02-17T12:18:45.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:84: create_scenario: rados -p test mksnap snap7 2026-02-17T12:18:45.418 INFO:tasks.workunit.client.0.trial128.stdout:created pool test snap snap7 2026-02-17T12:18:45.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:86: create_scenario: rados -p test rm obj4 2026-02-17T12:18:45.443 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:87: create_scenario: rados -p test rm obj16 2026-02-17T12:18:45.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:88: create_scenario: rados -p test rm obj2 2026-02-17T12:18:45.498 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:90: create_scenario: kill_daemons td/osd-scrub-snaps TERM osd 2026-02-17T12:18:45.499 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:18:45.499 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:18:45.499 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:18:45.499 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:18:45.499 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:18:46.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:18:46.810 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj1 2026-02-17T12:18:47.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:94: create_scenario: JSON='["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:18:47.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:95: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj1","key":"","snapid":-2,"hash":1828249343,"max":0,"pool":1,"namespace":"","max":0}]' --force remove 2026-02-17T12:18:48.329 INFO:tasks.workunit.client.0.trial128.stdout:WARNING: only removing head with clones present 2026-02-17T12:18:48.330 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:ff7b1f36:::obj1:head# 2026-02-17T12:18:48.867 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-17T12:18:48.867 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: grep '"snapid":2' 2026-02-17T12:18:49.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:97: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:18:49.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:98: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":2,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T12:18:50.372 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:c52c9666:::obj5:2# 2026-02-17T12:18:50.911 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-17T12:18:50.911 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: grep '"snapid":1' 2026-02-17T12:18:51.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:100: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:18:51.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:101: create_scenario: OBJ5SAVE='["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:18:51.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:103: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-17T12:18:52.840 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:104: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T12:18:52.841 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:18:52.841 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T12:18:52.842 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:18:52.842 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:18:52.842 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T12:18:52.842 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T12:18:52.842 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T12:18:52.842 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T12:18:52.842 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T12:18:52.842 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T12:18:52.842 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:18:52.842 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T12:18:52.843 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T12:18:52.843 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T12:18:52.843 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T12:18:52.843 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:18:52.843 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:18:52.843 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:18:52.843 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:18:52.843 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:18:52.843 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:18:52.843 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:18:52.843 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:18:52.843 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:18:52.844 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:18:52.844 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:18:52.844 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:18:52.844 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:18:52.844 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:18:52.844 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:18:52.844 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:18:52.844 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:18:52.844 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:18:52.844 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:18:52.844 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:18:52.845 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:18:52.845 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:18:52.845 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:18:52.845 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:18:52.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:105: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T12:18:52.846 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:18:52.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:106: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --rmtype nosnapmap '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T12:18:53.492 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:c52c9666:::obj5:1# 2026-02-17T12:18:54.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:108: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-17T12:18:55.119 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:109: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T12:18:55.121 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:18:55.121 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T12:18:55.121 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:18:55.121 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:18:55.121 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T12:18:55.121 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T12:18:55.121 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T12:18:55.121 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T12:18:55.122 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T12:18:55.122 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T12:18:55.122 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:18:55.122 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T12:18:55.122 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T12:18:55.122 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T12:18:55.122 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T12:18:55.122 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:18:55.122 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:18:55.122 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:18:55.122 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:18:55.122 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:18:55.123 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:18:55.123 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:18:55.123 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:18:55.123 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:18:55.123 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:18:55.123 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:18:55.123 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:18:55.123 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:18:55.123 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:18:55.123 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:18:55.123 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:18:55.123 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:18:55.123 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:18:55.124 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:18:55.124 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:18:55.124 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:18:55.124 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.0779578A.7.obj4.. 2026-02-17T12:18:55.124 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:18:55.124 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:18:55.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:110: create_scenario: grep '^[pm].*SNA_.*[.]1[.]obj5[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T12:18:55.125 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:18:55.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:111: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-17T12:18:55.125 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj5 2026-02-17T12:18:55.125 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: grep '"snapid":4' 2026-02-17T12:18:55.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:113: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:18:55.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:114: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=18 2026-02-17T12:18:55.978 INFO:tasks.workunit.client.0.trial128.stderr:18+0 records in 2026-02-17T12:18:55.978 INFO:tasks.workunit.client.0.trial128.stderr:18+0 records out 2026-02-17T12:18:55.979 INFO:tasks.workunit.client.0.trial128.stderr:4608 bytes (4.6 kB, 4.5 KiB) copied, 0.000253241 s, 18.2 MB/s 2026-02-17T12:18:55.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:115: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":4,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.452141 2026-02-17T12:18:57.161 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj3 2026-02-17T12:18:58.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:117: create_scenario: JSON='["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:18:58.012 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:118: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=15 2026-02-17T12:18:58.013 INFO:tasks.workunit.client.0.trial128.stderr:15+0 records in 2026-02-17T12:18:58.013 INFO:tasks.workunit.client.0.trial128.stderr:15+0 records out 2026-02-17T12:18:58.013 INFO:tasks.workunit.client.0.trial128.stderr:3840 bytes (3.8 kB, 3.8 KiB) copied, 9.3875e-05 s, 40.9 MB/s 2026-02-17T12:18:58.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:119: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj3","key":"","snapid":-2,"hash":1643547569,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.452141 2026-02-17T12:18:59.194 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj4 2026-02-17T12:18:59.195 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: grep '"snapid":7' 2026-02-17T12:19:00.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:121: create_scenario: JSON='["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:19:00.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:122: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj4","key":"","snapid":7,"hash":2826278768,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T12:19:00.691 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:0ee9ae15:::obj4:7# 2026-02-17T12:19:01.225 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:125: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-17T12:19:02.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:126: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T12:19:02.309 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T12:19:02.309 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:19:02.309 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:19:02.309 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T12:19:02.309 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T12:19:02.309 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T12:19:02.309 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T12:19:02.309 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T12:19:02.310 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T12:19:02.310 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:19:02.310 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T12:19:02.310 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T12:19:02.310 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T12:19:02.310 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T12:19:02.310 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:19:02.310 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:19:02.310 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:19:02.310 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:19:02.310 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:19:02.310 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:19:02.311 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:19:02.311 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:19:02.311 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:19:02.311 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:19:02.311 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:19:02.311 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:19:02.311 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:19:02.311 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:19:02.311 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:19:02.311 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:19:02.311 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:19:02.311 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:19:02.312 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:127: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T12:19:02.312 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:19:02.312 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:19:02.313 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:19:02.313 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:19:02.313 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:19:02.313 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:19:02.313 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.268F1DA7.7.obj16.. 2026-02-17T12:19:02.313 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --op list obj16 2026-02-17T12:19:02.313 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: grep '"snapid":7' 2026-02-17T12:19:03.156 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:128: create_scenario: JSON='["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:19:03.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:129: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --rmtype snapmap '["1.0",{"oid":"obj16","key":"","snapid":7,"hash":2060580962,"max":0,"pool":1,"namespace":"","max":0}]' remove 2026-02-17T12:19:03.787 INFO:tasks.workunit.client.0.trial128.stdout:remove #1:461f8b5e:::obj16:7# 2026-02-17T12:19:04.321 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:131: create_scenario: ceph-kvstore-tool bluestore-kv td/osd-scrub-snaps/0 list 2026-02-17T12:19:05.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:132: create_scenario: grep SNA_ td/osd-scrub-snaps/drk.log 2026-02-17T12:19:05.405 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.1BB86F16.1.obj3.. 2026-02-17T12:19:05.405 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.3A439666.1.obj5.. 2026-02-17T12:19:05.405 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.4CC52438.1.obj6.. 2026-02-17T12:19:05.405 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5685B6C7.1.obj13.. 2026-02-17T12:19:05.405 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5AAE7FD8.1.obj8.. 2026-02-17T12:19:05.405 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.5FAF9A3D.1.obj12.. 2026-02-17T12:19:05.405 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.70522149.1.obj14.. 2026-02-17T12:19:05.405 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.77574592.1.obj11.. 2026-02-17T12:19:05.405 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:19:05.406 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.950988C5.1.obj7.. 2026-02-17T12:19:05.406 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.B8CCECA2.1.obj10.. 2026-02-17T12:19:05.406 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.F7CA874E.1.obj9.. 2026-02-17T12:19:05.406 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000001_0000000000000001.FFED8FC6.1.obj1.. 2026-02-17T12:19:05.406 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:19:05.406 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000002_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:19:05.406 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.1BB86F16.3.obj3.. 2026-02-17T12:19:05.406 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:19:05.406 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000003_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:19:05.406 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.3A439666.4.obj5.. 2026-02-17T12:19:05.407 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000004_0000000000000001.802EE1F3.4.obj2.. 2026-02-17T12:19:05.407 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:19:05.407 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000005_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:19:05.407 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.3A439666.6.obj5.. 2026-02-17T12:19:05.407 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000006_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:19:05.407 INFO:tasks.workunit.client.0.trial128.stdout:p %00%00%00%00%00%00%00%00%c07%16%25%00%00%00%00%00%00%04%02.SNA_1_0000000000000007_0000000000000001.802EE1F3.7.obj2.. 2026-02-17T12:19:05.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:133: create_scenario: grep '^[pm].*SNA_.*[.]7[.]obj16[.][.]$' td/osd-scrub-snaps/drk.log 2026-02-17T12:19:05.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:134: create_scenario: rm -f td/osd-scrub-snaps/drk.log 2026-02-17T12:19:05.408 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj2 2026-02-17T12:19:06.260 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:136: create_scenario: JSON='["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:19:06.260 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:137: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj2","key":"","snapid":-2,"hash":1058988552,"max":0,"pool":1,"namespace":"","max":0}]' rm-attr snapset 2026-02-17T12:19:07.422 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: echo '["1.0",{"oid":"obj5","key":"","snapid":1,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:19:07.423 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: sed 's/snapid":1/snapid":7/' 2026-02-17T12:19:07.424 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:140: create_scenario: JSON='["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:19:07.424 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:141: create_scenario: dd if=/dev/urandom of=testdata.452141 bs=256 count=7 2026-02-17T12:19:07.425 INFO:tasks.workunit.client.0.trial128.stderr:7+0 records in 2026-02-17T12:19:07.425 INFO:tasks.workunit.client.0.trial128.stderr:7+0 records out 2026-02-17T12:19:07.425 INFO:tasks.workunit.client.0.trial128.stderr:1792 bytes (1.8 kB, 1.8 KiB) copied, 6.3228e-05 s, 28.3 MB/s 2026-02-17T12:19:07.425 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:142: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj5","key":"","snapid":7,"hash":1718170787,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes testdata.452141 2026-02-17T12:19:08.594 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj6 2026-02-17T12:19:08.912 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:19:09.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:144: create_scenario: JSON='["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:19:09.448 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:145: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj6","key":"","snapid":-2,"hash":2202164420,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset 2026-02-17T12:19:10.618 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj7 2026-02-17T12:19:10.938 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:19:11.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:146: create_scenario: JSON='["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:19:11.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:147: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj7","key":"","snapid":-2,"hash":1552453721,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset corrupt 2026-02-17T12:19:12.646 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj8 2026-02-17T12:19:12.956 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:19:13.495 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:148: create_scenario: JSON='["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:19:13.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:149: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj8","key":"","snapid":-2,"hash":2381834917,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset seq 2026-02-17T12:19:14.690 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj9 2026-02-17T12:19:15.007 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:19:15.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:150: create_scenario: JSON='["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:19:15.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:151: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj9","key":"","snapid":-2,"hash":3833113727,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_size 2026-02-17T12:19:16.722 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj10 2026-02-17T12:19:17.040 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:19:17.576 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:152: create_scenario: JSON='["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:19:17.576 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:153: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj10","key":"","snapid":-2,"hash":718195851,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clone_overlap 2026-02-17T12:19:18.741 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj11 2026-02-17T12:19:19.050 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:19:19.583 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:154: create_scenario: JSON='["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:19:19.583 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:155: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj11","key":"","snapid":-2,"hash":693400951,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset clones 2026-02-17T12:19:20.758 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj12 2026-02-17T12:19:21.067 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:19:21.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:156: create_scenario: JSON='["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:19:21.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:157: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj12","key":"","snapid":-2,"hash":3551132405,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset head 2026-02-17T12:19:22.770 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj13 2026-02-17T12:19:23.087 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:19:23.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:158: create_scenario: JSON='["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:19:23.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:159: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj13","key":"","snapid":-2,"hash":2087409765,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset snaps 2026-02-17T12:19:24.834 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj14 2026-02-17T12:19:25.144 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:19:25.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:160: create_scenario: JSON='["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:19:25.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:161: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj14","key":"","snapid":-2,"hash":2484217095,"max":0,"pool":1,"namespace":"","max":0}]' clear-snapset size 2026-02-17T12:19:26.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:163: create_scenario: echo garbage 2026-02-17T12:19:26.889 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 --head --op list obj15 2026-02-17T12:19:27.199 INFO:tasks.workunit.client.0.trial128.stderr:Error getting attr on : 1.0_head,#1:c52c9666:::obj5:7#, (61) No data available 2026-02-17T12:19:27.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:164: create_scenario: JSON='["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:19:27.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:165: create_scenario: ceph-objectstore-tool --data-path td/osd-scrub-snaps/0 '["1.0",{"oid":"obj15","key":"","snapid":-2,"hash":612772309,"max":0,"pool":1,"namespace":"","max":0}]' set-attr snapset td/osd-scrub-snaps/bad 2026-02-17T12:19:28.909 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:166: create_scenario: rm -f td/osd-scrub-snaps/bad 2026-02-17T12:19:28.910 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:167: create_scenario: return 0 2026-02-17T12:19:28.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:767: _scrub_snaps_multi: rm -f testdata.452141 2026-02-17T12:19:28.912 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: expr 2 - 1 2026-02-17T12:19:28.913 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: seq 0 1 2026-02-17T12:19:28.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:19:28.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:771: _scrub_snaps_multi: activate_osd td/osd-scrub-snaps 0 2026-02-17T12:19:28.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-snaps 2026-02-17T12:19:28.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:19:28.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-17T12:19:28.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:19:28.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-snaps/0 2026-02-17T12:19:28.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T12:19:28.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:19:28.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:19:28.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:19:28.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/0' 2026-02-17T12:19:28.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/0/journal' 2026-02-17T12:19:28.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:19:28.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:19:28.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T12:19:28.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:19:28.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:19:28.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:19:28.919 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:19:28.919 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:28.919 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:19:28.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:19:28.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:19:28.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:19:28.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T12:19:28.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T12:19:28.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:19:28.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:19:28.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:19:28.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:19:28.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:19:28.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:19:28.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-snaps/0 2026-02-17T12:19:28.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-17T12:19:28.921 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:19:28.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/0 --osd-journal=td/osd-scrub-snaps/0/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:19:28.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-snaps/0/whoami 2026-02-17T12:19:28.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-17T12:19:28.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:19:28.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:19:28.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:19:28.943 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:19:28.936+0000 7f272d2f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:28.945 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:19:28.936+0000 7f272d2f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:28.946 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:19:28.936+0000 7f272d2f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:29.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-17T12:19:29.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:19:29.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:19:29.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:19:29.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:19:29.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:29.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:19:29.083 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:19:29.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:29.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:19:29.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:30.201 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:19:30.192+0000 7f272d2f38c0 -1 Falling back to public interface 2026-02-17T12:19:30.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:30.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:30.244 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:19:30.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:19:30.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:30.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:19:30.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:31.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:31.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:31.401 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:19:31.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:19:31.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:31.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:19:31.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:32.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:32.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:32.564 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:19:32.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:19:32.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:32.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:19:32.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:33.196 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:19:33.188+0000 7f272d2f38c0 -1 osd.0 22 log_to_monitors true 2026-02-17T12:19:33.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:33.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:33.725 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:19:33.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:19:33.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:33.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:19:33.887 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 25 up_thru 25 down_at 23 last_clean_interval [5,22) [v2:127.0.0.1:6802/2124647350,v1:127.0.0.1:6803/2124647350] [v2:127.0.0.1:6804/2124647350,v1:127.0.0.1:6805/2124647350] exists,up ff2b90b2-0844-4470-b067-1d09fd2f612d 2026-02-17T12:19:33.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:19:33.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:19:33.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:19:33.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:769: _scrub_snaps_multi: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:19:33.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:771: _scrub_snaps_multi: activate_osd td/osd-scrub-snaps 1 2026-02-17T12:19:33.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-snaps 2026-02-17T12:19:33.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:19:33.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:19:33.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:19:33.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-snaps/1 2026-02-17T12:19:33.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1' 2026-02-17T12:19:33.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:19:33.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:19:33.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:19:33.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-snaps/1' 2026-02-17T12:19:33.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-snaps/1/journal' 2026-02-17T12:19:33.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:19:33.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:19:33.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-snaps' 2026-02-17T12:19:33.893 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:19:33.893 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:19:33.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:19:33.894 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:19:33.894 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:33.894 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:19:33.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:19:33.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' 2026-02-17T12:19:33.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:19:33.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-snaps/$name.log' 2026-02-17T12:19:33.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-snaps/$name.pid' 2026-02-17T12:19:33.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:19:33.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:19:33.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:19:33.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:19:33.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:19:33.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:19:33.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-snaps/1 2026-02-17T12:19:33.895 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:19:33.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:19:33.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 --osd_scrub_chunk_min=3 --osd_scrub_chunk_max=20 --osd_shallow_scrub_chunk_min=3 --osd_shallow_scrub_chunk_max=3 --osd_pg_stat_report_interval_max_seconds=1 --osd_pg_stat_report_interval_max_epochs=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-snaps/1 --osd-journal=td/osd-scrub-snaps/1/journal --chdir= --run-dir=td/osd-scrub-snaps '--admin-socket=/tmp/ceph-asok.452141/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-snaps/$name.log' '--pid-file=td/osd-scrub-snaps/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:19:33.897 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-snaps/1/whoami 2026-02-17T12:19:33.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:19:33.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:19:33.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:19:33.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:19:33.916 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:19:33.908+0000 7fc1a46e88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:33.917 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:19:33.908+0000 7fc1a46e88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:33.918 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:19:33.908+0000 7fc1a46e88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:34.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:19:34.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:19:34.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:19:34.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:19:34.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:19:34.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:34.047 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:19:34.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:19:34.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:34.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:34.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:35.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:35.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:35.203 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:19:35.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:19:35.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:35.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:35.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:35.405 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:19:35.396+0000 7fc1a46e88c0 -1 Falling back to public interface 2026-02-17T12:19:36.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:36.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:36.361 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:19:36.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:19:36.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:36.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:36.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:37.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:37.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:37.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:19:37.517 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:19:37.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:37.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:37.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:19:38.419 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:19:38.408+0000 7fc1a46e88c0 -1 osd.1 22 log_to_monitors true 2026-02-17T12:19:38.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:19:38.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:19:38.676 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:19:38.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:19:38.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:19:38.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:19:38.834 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 28 up_thru 28 down_at 23 last_clean_interval [10,22) [v2:127.0.0.1:6810/1952186744,v1:127.0.0.1:6811/1952186744] [v2:127.0.0.1:6812/1952186744,v1:127.0.0.1:6813/1952186744] exists,up 98ad9e01-ae90-4107-a892-afc1bd515b4c 2026-02-17T12:19:38.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:19:38.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:19:38.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:19:38.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:774: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_max 3 2026-02-17T12:19:38.904 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T12:19:38.904 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_shallow_scrub_chunk_max = '' (not observed, change may require restart) osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-17T12:19:38.904 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:19:38.910 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T12:19:38.910 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_shallow_scrub_chunk_max = '' (not observed, change may require restart) osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-17T12:19:38.911 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:19:38.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:775: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_min 3 2026-02-17T12:19:38.996 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T12:19:38.996 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_shallow_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T12:19:38.996 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:19:39.002 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T12:19:39.002 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_shallow_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T12:19:39.002 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:19:39.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:776: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_scrub_chunk_min 3 2026-02-17T12:19:39.084 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T12:19:39.084 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T12:19:39.085 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:19:39.091 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T12:19:39.091 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_chunk_min = '' (not observed, change may require restart) " 2026-02-17T12:19:39.091 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:19:39.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:777: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_seconds 1 2026-02-17T12:19:39.176 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T12:19:39.176 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_pg_stat_report_interval_max_seconds = '' (not observed, change may require restart) " 2026-02-17T12:19:39.176 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:19:39.182 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T12:19:39.182 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_pg_stat_report_interval_max_seconds = '' (not observed, change may require restart) " 2026-02-17T12:19:39.182 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:19:39.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:778: _scrub_snaps_multi: ceph tell 'osd.*' config set osd_pg_stat_report_interval_max_epochs 1 2026-02-17T12:19:39.261 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T12:19:39.261 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_pg_stat_report_interval_max_epochs = '' (not observed, change may require restart) " 2026-02-17T12:19:39.262 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:19:39.268 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T12:19:39.268 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_pg_stat_report_interval_max_epochs = '' (not observed, change may require restart) " 2026-02-17T12:19:39.268 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:19:39.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:779: _scrub_snaps_multi: wait_for_clean 2026-02-17T12:19:39.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:19:39.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:19:39.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:19:39.276 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:19:39.277 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:19:39.277 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:19:39.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:19:39.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:19:39.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:19:39.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:19:39.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:19:39.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:19:39.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:19:39.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:19:39.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:19:39.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:19:39.549 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:19:39.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:19:39.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:39.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:19:39.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182403 2026-02-17T12:19:39.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182403 2026-02-17T12:19:39.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182403' 2026-02-17T12:19:39.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:39.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:19:39.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=120259084290 2026-02-17T12:19:39.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 120259084290 2026-02-17T12:19:39.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182403 1-120259084290' 2026-02-17T12:19:39.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:39.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-107374182403 2026-02-17T12:19:39.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:39.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:19:39.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-107374182403 2026-02-17T12:19:39.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:39.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182403 2026-02-17T12:19:39.705 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 107374182403 2026-02-17T12:19:39.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 107374182403' 2026-02-17T12:19:39.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:39.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182402 -lt 107374182403 2026-02-17T12:19:39.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:19:40.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:19:40.865 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:41.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182402 -lt 107374182403 2026-02-17T12:19:41.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:19:42.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:19:42.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:42.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182403 -lt 107374182403 2026-02-17T12:19:42.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:42.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-120259084290 2026-02-17T12:19:42.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:42.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:19:42.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-120259084290 2026-02-17T12:19:42.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:42.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=120259084290 2026-02-17T12:19:42.189 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 120259084290 2026-02-17T12:19:42.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 120259084290' 2026-02-17T12:19:42.189 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:19:42.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 120259084290 -lt 120259084290 2026-02-17T12:19:42.351 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:19:42.351 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:19:42.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:19:42.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:19:42.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:19:42.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:19:42.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:19:42.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:19:42.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:19:42.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:19:42.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:19:42.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:19:42.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:19:42.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:19:42.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:19:42.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:19:42.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:19:42.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:19:42.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:781: _scrub_snaps_multi: local pgid=1.0 2026-02-17T12:19:42.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:782: _scrub_snaps_multi: pg_scrub 1.0 2026-02-17T12:19:42.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=1.0 2026-02-17T12:19:42.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 1.0 2026-02-17T12:19:42.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=1.0 2026-02-17T12:19:42.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T12:19:42.889 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:19:42.889 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:19:42.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:19:42.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:19:42.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:19:43.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T12:19:43.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T12:19:43.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T12:19:43.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T12:19:43.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:19:43.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:19:43.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:19:43.318 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:19:43.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:19:43.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:43.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:19:43.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=107374182405 2026-02-17T12:19:43.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 107374182405 2026-02-17T12:19:43.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182405' 2026-02-17T12:19:43.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:19:43.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:19:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=120259084292 2026-02-17T12:19:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 120259084292 2026-02-17T12:19:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-107374182405 1-120259084292' 2026-02-17T12:19:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:43.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-107374182405 2026-02-17T12:19:43.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:43.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:19:43.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-107374182405 2026-02-17T12:19:43.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:43.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=107374182405 2026-02-17T12:19:43.474 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 107374182405 2026-02-17T12:19:43.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 107374182405' 2026-02-17T12:19:43.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:43.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182403 -lt 107374182405 2026-02-17T12:19:43.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:19:44.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:19:44.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:19:44.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 107374182405 -lt 107374182405 2026-02-17T12:19:44.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:19:44.800 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-120259084292 2026-02-17T12:19:44.800 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:19:44.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:19:44.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-120259084292 2026-02-17T12:19:44.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:19:44.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=120259084292 2026-02-17T12:19:44.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 120259084292' 2026-02-17T12:19:44.804 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 120259084292 2026-02-17T12:19:44.805 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:19:44.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 120259084292 -lt 120259084292 2026-02-17T12:19:44.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T12:19:44.968 INFO:tasks.workunit.client.0.trial128.stdout:#---------- 1.0 loop 0 2026-02-17T12:19:44.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T12:19:44.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T12:19:44.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=1.0 2026-02-17T12:19:44.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T12:19:44.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 1.0 query 2026-02-17T12:19:44.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T12:19:45.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T12:19:45.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T12:19:45.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T12:19:45.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T12:19:45.044 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 1.0 2026-02-17T12:19:45.044 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:19:45.045 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:19:45.045 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:19:45.045 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:19:45.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-17T12:18:39.950839+0000 2026-02-17T12:19:45.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 1.0 2026-02-17T12:19:45.348 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 1.0 on osd.1 to scrub 2026-02-17T12:19:45.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 1.0 2026-02-17T12:18:39.950839+0000 2026-02-17T12:19:45.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T12:19:45.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:18:39.950839+0000 2026-02-17T12:19:45.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:19:45.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:19:45.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:19:45.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:19:45.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:19:45.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:19:45.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:19:45.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:19:45.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:18:39.950839+0000 '>' 2026-02-17T12:18:39.950839+0000 2026-02-17T12:19:45.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:19:46.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:19:46.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:19:46.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:19:46.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:19:46.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:19:46.519 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:19:46.519 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:19:46.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:18:39.950839+0000 '>' 2026-02-17T12:18:39.950839+0000 2026-02-17T12:19:46.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:19:47.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:19:47.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:19:47.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:19:47.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:19:47.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:19:47.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:19:47.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:19:47.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:18:39.950839+0000 '>' 2026-02-17T12:18:39.950839+0000 2026-02-17T12:19:47.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:19:48.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:19:48.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:19:48.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:19:48.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:19:48.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:19:48.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:19:48.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:19:48.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:18:39.950839+0000 '>' 2026-02-17T12:18:39.950839+0000 2026-02-17T12:19:48.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:19:49.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:19:49.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 300 )) 2026-02-17T12:19:49.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:19:49.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:19:49.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:19:49.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:19:49.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:19:50.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:19:45.418573+0000 '>' 2026-02-17T12:18:39.950839+0000 2026-02-17T12:19:50.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:19:50.156 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: grep '_scan_snaps start' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:19:50.156 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: wc -l 2026-02-17T12:19:50.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:786: _scrub_snaps_multi: test 16 -gt 3 2026-02-17T12:19:50.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: grep '_scan_snaps start' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:19:50.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: wc -l 2026-02-17T12:19:50.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:787: _scrub_snaps_multi: test 16 -gt 3 2026-02-17T12:19:50.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:789: _scrub_snaps_multi: rados list-inconsistent-pg test 2026-02-17T12:19:50.184 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:791: _scrub_snaps_multi: jq '. | length' td/osd-scrub-snaps/json 2026-02-17T12:19:50.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:791: _scrub_snaps_multi: test 1 = 1 2026-02-17T12:19:50.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:793: _scrub_snaps_multi: jq -r '.[0]' td/osd-scrub-snaps/json 2026-02-17T12:19:50.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:793: _scrub_snaps_multi: test 1.0 = 1.0 2026-02-17T12:19:50.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:795: _scrub_snaps_multi: rados list-inconsistent-obj 1.0 --format=json-pretty 2026-02-17T12:19:50.220 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:19:50.221 INFO:tasks.workunit.client.0.trial128.stdout: "epoch": 28, 2026-02-17T12:19:50.221 INFO:tasks.workunit.client.0.trial128.stdout: "inconsistents": [ 2026-02-17T12:19:50.221 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.221 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.221 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj5", 2026-02-17T12:19:50.221 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.221 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.221 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:19:50.221 INFO:tasks.workunit.client.0.trial128.stdout: "version": 5 2026-02-17T12:19:50.221 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.222 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.222 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:19:50.222 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:19:50.222 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.222 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:19:50.222 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.222 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj5", 2026-02-17T12:19:50.222 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.222 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": 1, 2026-02-17T12:19:50.222 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1718170787, 2026-02-17T12:19:50.222 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.223 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.223 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.223 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.223 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'19", 2026-02-17T12:19:50.223 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'5", 2026-02-17T12:19:50.223 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4189.0:1", 2026-02-17T12:19:50.223 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 5, 2026-02-17T12:19:50.223 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.223 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:43.704084+0000", 2026-02-17T12:19:50.223 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:43.704418+0000", 2026-02-17T12:19:50.223 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.223 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.224 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.224 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.224 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.224 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.224 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.224 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x8adb1815", 2026-02-17T12:19:50.224 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.224 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.224 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.224 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.224 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.224 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.225 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.225 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.225 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.225 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.225 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.225 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.225 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.225 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.225 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:19:50.225 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.225 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.225 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.226 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.226 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.226 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.226 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032 2026-02-17T12:19:50.226 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.226 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.226 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.226 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.226 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.226 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj5", 2026-02-17T12:19:50.226 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.226 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.227 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 2, 2026-02-17T12:19:50.227 INFO:tasks.workunit.client.0.trial128.stdout: "version": 20 2026-02-17T12:19:50.227 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.227 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.227 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:19:50.227 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:19:50.227 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.227 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:19:50.227 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.227 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj5", 2026-02-17T12:19:50.227 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.227 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": 2, 2026-02-17T12:19:50.228 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1718170787, 2026-02-17T12:19:50.228 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.228 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.228 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.228 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.228 INFO:tasks.workunit.client.0.trial128.stdout: "version": "17'41", 2026-02-17T12:19:50.228 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "16'20", 2026-02-17T12:19:50.228 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4237.0:1", 2026-02-17T12:19:50.228 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 20, 2026-02-17T12:19:50.228 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.228 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:44.563038+0000", 2026-02-17T12:19:50.228 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:44.563473+0000", 2026-02-17T12:19:50.228 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.229 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.229 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.229 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.229 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.229 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.229 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.229 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x543c42db", 2026-02-17T12:19:50.229 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.229 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.229 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.229 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.229 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.230 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.230 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.230 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.230 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.230 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.230 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.230 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.230 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.230 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.230 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:19:50.230 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.230 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.230 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.231 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.231 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.231 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.231 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256 2026-02-17T12:19:50.231 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.231 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.231 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.231 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.231 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.231 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj5", 2026-02-17T12:19:50.231 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.231 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.232 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 4, 2026-02-17T12:19:50.232 INFO:tasks.workunit.client.0.trial128.stdout: "version": 42 2026-02-17T12:19:50.232 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.232 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.232 INFO:tasks.workunit.client.0.trial128.stdout: "size_mismatch" 2026-02-17T12:19:50.232 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.232 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:19:50.232 INFO:tasks.workunit.client.0.trial128.stdout: "size_mismatch_info", 2026-02-17T12:19:50.232 INFO:tasks.workunit.client.0.trial128.stdout: "obj_size_info_mismatch" 2026-02-17T12:19:50.232 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.232 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:19:50.232 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.232 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj5", 2026-02-17T12:19:50.233 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.233 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": 4, 2026-02-17T12:19:50.233 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1718170787, 2026-02-17T12:19:50.233 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.233 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.233 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.233 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.233 INFO:tasks.workunit.client.0.trial128.stdout: "version": "19'45", 2026-02-17T12:19:50.233 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "17'42", 2026-02-17T12:19:50.233 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4273.0:1", 2026-02-17T12:19:50.233 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 42, 2026-02-17T12:19:50.233 INFO:tasks.workunit.client.0.trial128.stdout: "size": 512, 2026-02-17T12:19:50.234 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:44.932640+0000", 2026-02-17T12:19:50.234 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:44.934115+0000", 2026-02-17T12:19:50.234 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.234 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.234 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.234 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.234 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.234 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.234 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.234 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xbbcd5635", 2026-02-17T12:19:50.234 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.234 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.234 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.235 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.235 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.235 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.235 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.235 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.235 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.235 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.235 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.235 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.235 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.235 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.236 INFO:tasks.workunit.client.0.trial128.stdout: "size_mismatch_info", 2026-02-17T12:19:50.236 INFO:tasks.workunit.client.0.trial128.stdout: "obj_size_info_mismatch" 2026-02-17T12:19:50.236 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.236 INFO:tasks.workunit.client.0.trial128.stdout: "size": 4608, 2026-02-17T12:19:50.236 INFO:tasks.workunit.client.0.trial128.stdout: "object_info": { 2026-02-17T12:19:50.236 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.236 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj5", 2026-02-17T12:19:50.236 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.236 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": 4, 2026-02-17T12:19:50.236 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1718170787, 2026-02-17T12:19:50.236 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.236 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.237 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.237 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.237 INFO:tasks.workunit.client.0.trial128.stdout: "version": "19'45", 2026-02-17T12:19:50.237 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "17'42", 2026-02-17T12:19:50.237 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4273.0:1", 2026-02-17T12:19:50.237 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 42, 2026-02-17T12:19:50.237 INFO:tasks.workunit.client.0.trial128.stdout: "size": 512, 2026-02-17T12:19:50.237 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:44.932640+0000", 2026-02-17T12:19:50.237 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:44.934115+0000", 2026-02-17T12:19:50.237 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.237 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.237 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.238 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.238 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.238 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.238 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.238 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xbbcd5635", 2026-02-17T12:19:50.238 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.238 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.238 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.238 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.238 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.238 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.238 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.239 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.239 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.239 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.239 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.239 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.239 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.239 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.239 INFO:tasks.workunit.client.0.trial128.stdout: "size": 512 2026-02-17T12:19:50.239 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.239 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.239 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.240 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj4", 2026-02-17T12:19:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 7, 2026-02-17T12:19:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "version": 4 2026-02-17T12:19:50.240 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:19:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:19:50.240 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:19:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj4", 2026-02-17T12:19:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": 7, 2026-02-17T12:19:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 2826278768, 2026-02-17T12:19:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.241 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "version": "22'51", 2026-02-17T12:19:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'4", 2026-02-17T12:19:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4186.0:1", 2026-02-17T12:19:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 4, 2026-02-17T12:19:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:43.676170+0000", 2026-02-17T12:19:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:43.676571+0000", 2026-02-17T12:19:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.242 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x8adb1815", 2026-02-17T12:19:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.243 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.243 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.243 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.244 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.244 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.244 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.244 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:19:50.244 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.244 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.244 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.244 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.244 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.244 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.244 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032 2026-02-17T12:19:50.245 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.245 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.245 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.245 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj5", 2026-02-17T12:19:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 7, 2026-02-17T12:19:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "version": 0 2026-02-17T12:19:50.245 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.246 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:19:50.246 INFO:tasks.workunit.client.0.trial128.stdout: "missing", 2026-02-17T12:19:50.246 INFO:tasks.workunit.client.0.trial128.stdout: "info_missing" 2026-02-17T12:19:50.246 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.246 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.246 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.246 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.246 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.246 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.246 INFO:tasks.workunit.client.0.trial128.stdout: "info_missing" 2026-02-17T12:19:50.246 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.247 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1792 2026-02-17T12:19:50.247 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.247 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.247 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.247 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.247 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.247 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:19:50.247 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.247 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.247 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.247 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.247 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj1", 2026-02-17T12:19:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:19:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "version": 18 2026-02-17T12:19:50.248 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:19:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:19:50.248 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:19:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj1", 2026-02-17T12:19:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:19:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1828249343, 2026-02-17T12:19:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.249 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'18", 2026-02-17T12:19:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'1", 2026-02-17T12:19:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4234.0:1", 2026-02-17T12:19:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 18, 2026-02-17T12:19:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:44.535511+0000", 2026-02-17T12:19:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:44.535912+0000", 2026-02-17T12:19:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.250 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x543c42db", 2026-02-17T12:19:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.251 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.251 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.251 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.252 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.252 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.252 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.252 INFO:tasks.workunit.client.0.trial128.stdout: "missing" 2026-02-17T12:19:50.252 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.252 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.252 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.252 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.252 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.252 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.252 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256 2026-02-17T12:19:50.252 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.253 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.253 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.253 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.253 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.253 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj10", 2026-02-17T12:19:50.253 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.253 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.253 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:19:50.253 INFO:tasks.workunit.client.0.trial128.stdout: "version": 32 2026-02-17T12:19:50.253 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.253 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.253 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_inconsistency" 2026-02-17T12:19:50.254 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.254 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [], 2026-02-17T12:19:50.254 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:19:50.254 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.254 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj10", 2026-02-17T12:19:50.254 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.254 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:19:50.254 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 718195851, 2026-02-17T12:19:50.254 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.254 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.254 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.254 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.255 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'32", 2026-02-17T12:19:50.255 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'10", 2026-02-17T12:19:50.255 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4255.0:1", 2026-02-17T12:19:50.255 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 32, 2026-02-17T12:19:50.255 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.255 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:44.718436+0000", 2026-02-17T12:19:50.255 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:44.718799+0000", 2026-02-17T12:19:50.255 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.255 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.255 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.255 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.255 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.255 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.256 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.256 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x543c42db", 2026-02-17T12:19:50.256 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.256 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.256 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.256 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.256 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.256 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.256 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.256 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.256 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.256 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.256 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.257 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.257 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.257 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.257 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.257 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.257 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:19:50.257 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:19:50.257 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.257 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:19:50.257 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.257 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "????", 2026-02-17T12:19:50.257 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:19:50.257 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:19:50.257 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.258 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.258 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.258 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.258 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.258 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.258 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.258 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.258 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.258 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.258 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.258 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:19:50.258 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:19:50.258 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.258 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:19:50.259 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.259 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:19:50.259 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:19:50.259 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:19:50.259 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.259 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.259 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.259 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.259 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.259 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.259 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.259 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.260 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.260 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj11", 2026-02-17T12:19:50.260 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.260 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.260 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:19:50.260 INFO:tasks.workunit.client.0.trial128.stdout: "version": 34 2026-02-17T12:19:50.260 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.260 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.260 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_inconsistency" 2026-02-17T12:19:50.260 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.260 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [], 2026-02-17T12:19:50.260 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:19:50.260 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.260 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj11", 2026-02-17T12:19:50.261 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.261 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:19:50.261 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 693400951, 2026-02-17T12:19:50.261 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.261 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.261 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.261 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.261 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'34", 2026-02-17T12:19:50.261 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'11", 2026-02-17T12:19:50.261 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4258.0:1", 2026-02-17T12:19:50.261 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 34, 2026-02-17T12:19:50.261 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.261 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:44.743835+0000", 2026-02-17T12:19:50.262 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:44.744194+0000", 2026-02-17T12:19:50.262 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.262 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.262 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.262 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.262 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.262 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.262 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.262 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x543c42db", 2026-02-17T12:19:50.262 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.262 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.262 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.262 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.262 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.263 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.263 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.263 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.263 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.263 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.263 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.263 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.263 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.263 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.263 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.263 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.263 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:19:50.263 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [] 2026-02-17T12:19:50.263 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.264 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.264 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.264 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.264 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.264 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.264 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.264 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.264 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:19:50.264 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:19:50.264 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.264 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:19:50.264 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.264 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:19:50.265 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:19:50.265 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:19:50.265 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.265 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.265 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.265 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.265 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.265 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.265 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.265 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.265 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.265 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj13", 2026-02-17T12:19:50.265 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.265 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: "version": 38 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_inconsistency" 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [], 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj13", 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 2087409765, 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.266 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'38", 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'13", 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4264.0:1", 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 38, 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:44.793580+0000", 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:44.793908+0000", 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x543c42db", 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.267 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.268 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.268 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.268 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.268 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.268 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.268 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.268 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.268 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.268 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.268 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.268 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.268 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.268 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.268 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:19:50.268 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:19:50.268 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]" 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.269 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj14", 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: "version": 40 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.270 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_inconsistency" 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [], 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj14", 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 2484217095, 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'40", 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'14", 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4267.0:1", 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 40, 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.271 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:44.821439+0000", 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:44.821881+0000", 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x543c42db", 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.272 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1033, 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.273 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj15", 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.274 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "version": 15 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_corrupted" 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj15", 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 612772309, 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "version": "15'15", 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "0'0", 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4219.0:1", 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 15, 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.275 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:43.980815+0000", 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:43.981112+0000", 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x8adb1815", 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.276 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_corrupted" 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": "Z2FyYmFnZQo=" 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 0, 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [] 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.277 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj2", 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "version": 56 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_missing" 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj2", 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1058988552, 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "version": "22'56", 2026-02-17T12:19:50.278 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "19'48", 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4309.0:1", 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 56, 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "size": 0, 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:45.489191+0000", 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:45.489495+0000", 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "whiteout", 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "dirty" 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0xffffffff", 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.279 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_missing" 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: "size": 0 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: "size": 0, 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 7, 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 4, 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: 4, 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: 3, 2026-02-17T12:19:50.280 INFO:tasks.workunit.client.0.trial128.stdout: 2, 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 7, 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1024, 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: 7, 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: 6, 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: 5 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj3", 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:19:50.281 INFO:tasks.workunit.client.0.trial128.stdout: "version": 44 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "size_mismatch" 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [ 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "size_mismatch_info", 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "obj_size_info_mismatch" 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj3", 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1643547569, 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "version": "18'44", 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "16'22", 2026-02-17T12:19:50.282 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4279.0:1", 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 44, 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "size": 768, 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:45.035089+0000", 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:45.035575+0000", 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x779ce62e", 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.283 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "size_mismatch_info", 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "obj_size_info_mismatch" 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "size": 3840, 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "object_info": { 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj3", 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1643547569, 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "version": "18'44", 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "16'22", 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4279.0:1", 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 44, 2026-02-17T12:19:50.284 INFO:tasks.workunit.client.0.trial128.stdout: "size": 768, 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:45.035089+0000", 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:45.035575+0000", 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x779ce62e", 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: "size": 768 2026-02-17T12:19:50.285 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj6", 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "version": 24 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_inconsistency" 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [], 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj6", 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 2202164420, 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'24", 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'6", 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4243.0:1", 2026-02-17T12:19:50.286 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 24, 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:44.617819+0000", 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:44.618228+0000", 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x543c42db", 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.287 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [] 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.288 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj7", 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "version": 26 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_inconsistency" 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [], 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj7", 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 1552453721, 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'26", 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'7", 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4246.0:1", 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 26, 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:44.646757+0000", 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:44.647105+0000", 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.289 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x543c42db", 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.290 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 0, 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [] 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj8", 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.291 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "version": 28 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_inconsistency" 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [], 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj8", 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 2381834917, 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'28", 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'8", 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4249.0:1", 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 28, 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:44.674240+0000", 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:44.674527+0000", 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.292 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x543c42db", 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 0, 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:19:50.293 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.294 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "object": { 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "name": "obj9", 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "nspace": "", 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "locator": "", 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "snap": "head", 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "version": 30 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [ 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "snapset_inconsistency" 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "union_shard_errors": [], 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "selected_object_info": { 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "oid": { 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "oid": "obj9", 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "key": "", 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "snapid": -2, 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "hash": 3833113727, 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "max": 0, 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "pool": 1, 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "namespace": "" 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "version": "16'30", 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "prior_version": "15'9", 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "last_reqid": "client.4252.0:1", 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "user_version": 30, 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "mtime": "2026-02-17T12:18:44.698986+0000", 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "local_mtime": "2026-02-17T12:18:44.699315+0000", 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "lost": 0, 2026-02-17T12:19:50.295 INFO:tasks.workunit.client.0.trial128.stdout: "flags": [ 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "dirty", 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest" 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_seq": 0, 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "truncate_size": 0, 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "data_digest": "0x543c42db", 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "omap_digest": "0xffffffff", 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "expected_object_size": 0, 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "expected_write_size": 0, 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "alloc_hint_flags": 0, 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "manifest": { 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "type": 0 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "watchers": {} 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "shards": [ 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 0, 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "primary": false, 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.296 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:19:50.297 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:19:50.297 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.297 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:19:50.297 INFO:tasks.workunit.client.0.trial128.stdout: "size": "????", 2026-02-17T12:19:50.297 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:19:50.297 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:19:50.297 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:19:50.297 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.297 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.297 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.297 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.297 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:19:50.297 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.297 INFO:tasks.workunit.client.0.trial128.stdout: "osd": 1, 2026-02-17T12:19:50.297 INFO:tasks.workunit.client.0.trial128.stdout: "primary": true, 2026-02-17T12:19:50.297 INFO:tasks.workunit.client.0.trial128.stdout: "errors": [], 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: "size": 256, 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: "snapset": { 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: "seq": 1, 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: "clones": [ 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: "snap": 1, 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: "size": 1032, 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: "overlap": "[]", 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: "snaps": [ 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: 1 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:19:50.298 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:19:50.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:797: _scrub_snaps_multi: rados list-inconsistent-snapset 1.0 2026-02-17T12:19:50.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:800: _scrub_snaps_multi: '[' replica = replica ']' 2026-02-17T12:19:50.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:802: _scrub_snaps_multi: scruberrors=20 2026-02-17T12:19:50.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:803: _scrub_snaps_multi: jq .inconsistents 2026-02-17T12:19:50.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:803: _scrub_snaps_multi: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-17T12:19:50.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1066: _scrub_snaps_multi: jq .inconsistents td/osd-scrub-snaps/json 2026-02-17T12:19:50.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1066: _scrub_snaps_multi: python3 -c 'import json; import sys ; JSON=sys.stdin.read() ; ud = json.loads(JSON) ; print ( json.dumps(ud, sort_keys=True, indent=2) )' 2026-02-17T12:19:50.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1067: _scrub_snaps_multi: multidiff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-17T12:19:50.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: multidiff: diff td/osd-scrub-snaps/checkcsjson td/osd-scrub-snaps/csjson 2026-02-17T12:19:50.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1068: _scrub_snaps_multi: test no = yes 2026-02-17T12:19:50.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1073: _scrub_snaps_multi: test '' = yes 2026-02-17T12:19:50.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: find td/osd-scrub-snaps 2026-02-17T12:19:50.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: grep 'osd[^/]*\.pid' 2026-02-17T12:19:50.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1078: _scrub_snaps_multi: pidfiles='td/osd-scrub-snaps/osd.1.pid 2026-02-17T12:19:50.300 INFO:tasks.workunit.client.0.trial128.stderr:td/osd-scrub-snaps/osd.0.pid' 2026-02-17T12:19:50.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1079: _scrub_snaps_multi: pids= 2026-02-17T12:19:50.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1080: _scrub_snaps_multi: for pidfile in ${pidfiles} 2026-02-17T12:19:50.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: cat td/osd-scrub-snaps/osd.1.pid 2026-02-17T12:19:50.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: pids+='506416 ' 2026-02-17T12:19:50.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1080: _scrub_snaps_multi: for pidfile in ${pidfiles} 2026-02-17T12:19:50.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: cat td/osd-scrub-snaps/osd.0.pid 2026-02-17T12:19:50.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1082: _scrub_snaps_multi: pids+='505803 ' 2026-02-17T12:19:50.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1085: _scrub_snaps_multi: ERRORS=0 2026-02-17T12:19:50.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1089: _scrub_snaps_multi: '[' replica = primary ']' 2026-02-17T12:19:50.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1111: _scrub_snaps_multi: ceph pg dump pgs 2026-02-17T12:19:50.431 INFO:tasks.workunit.client.0.trial128.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-17T12:19:50.431 INFO:tasks.workunit.client.0.trial128.stdout:1.0 36 0 0 0 0 24448 0 0 56 0 56 active+clean+inconsistent 2026-02-17T12:19:45.418647+0000 22'56 30:104 [1,0] 1 [1,0] 1 22'56 2026-02-17T12:19:45.418573+0000 0'0 2026-02-17T12:18:39.950839+0000 0 1 periodic scrub scheduled @ 2026-02-18T13:16:40.211499+0000 36 0 2026-02-17T12:19:50.431 INFO:tasks.workunit.client.0.trial128.stdout: 2026-02-17T12:19:50.431 INFO:tasks.workunit.client.0.trial128.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-17T12:19:50.431 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:19:50.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1113: _scrub_snaps_multi: for pid in $pids 2026-02-17T12:19:50.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1115: _scrub_snaps_multi: kill -0 506416 2026-02-17T12:19:50.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1113: _scrub_snaps_multi: for pid in $pids 2026-02-17T12:19:50.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1115: _scrub_snaps_multi: kill -0 505803 2026-02-17T12:19:50.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1122: _scrub_snaps_multi: kill_daemons td/osd-scrub-snaps 2026-02-17T12:19:50.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:19:50.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:19:50.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:19:50.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:19:50.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:19:58.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:19:58.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1124: _scrub_snaps_multi: declare -a err_strings 2026-02-17T12:19:58.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1125: _scrub_snaps_multi: err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj4:7 : missing' 2026-02-17T12:19:58.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1126: _scrub_snaps_multi: err_strings[1]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj3:head : size 3840 != size 768 from auth oi' 2026-02-17T12:19:58.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1127: _scrub_snaps_multi: err_strings[2]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:1 : missing' 2026-02-17T12:19:58.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1128: _scrub_snaps_multi: err_strings[3]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:2 : missing' 2026-02-17T12:19:58.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1129: _scrub_snaps_multi: err_strings[4]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj5:4 : size 4608 != size 512 from auth oi' 2026-02-17T12:19:58.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1130: _scrub_snaps_multi: err_strings[5]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid .*:::obj5:7 : failed to pick suitable object info' 2026-02-17T12:19:58.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1131: _scrub_snaps_multi: err_strings[6]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj1:head : missing' 2026-02-17T12:19:58.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1132: _scrub_snaps_multi: err_strings[7]='log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 20 errors' 2026-02-17T12:19:58.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T12:19:58.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj4:7 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:19:58.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T12:19:58.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj3:head : size 3840 != size 768 from auth oi' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:19:58.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T12:19:58.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:1 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:19:58.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T12:19:58.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj5:2 : missing' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:19:58.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T12:19:58.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] soid .*:::obj5:4 : size 4608 != size 512 from auth oi' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:19:58.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T12:19:58.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 soid .*:::obj5:7 : failed to pick suitable object info' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:19:58.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T12:19:58.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 shard [0-1] .*:::obj1:head : missing' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:19:58.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1134: _scrub_snaps_multi: for err_string in "${err_strings[@]}" 2026-02-17T12:19:58.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1136: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : [0-9]*[.]0 scrub 20 errors' td/osd-scrub-snaps/osd.1.log 2026-02-17T12:19:58.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1144: _scrub_snaps_multi: declare -a rep_err_strings 2026-02-17T12:19:58.780 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: eval echo '$replica' 2026-02-17T12:19:58.780 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: echo 0 2026-02-17T12:19:58.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1145: _scrub_snaps_multi: osd=0 2026-02-17T12:19:58.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1146: _scrub_snaps_multi: rep_err_strings[0]='log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' 2026-02-17T12:19:58.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1147: _scrub_snaps_multi: for err_string in "${rep_err_strings[@]}" 2026-02-17T12:19:58.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1149: _scrub_snaps_multi: grep 'log_channel[(]cluster[)] log [[]ERR[]] : osd[.][0-9]* found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: {1, 2, 3, 4, 5, 6, 7} ...repaired' td/osd-scrub-snaps/osd.0.log 2026-02-17T12:19:58.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1156: _scrub_snaps_multi: '[' 0 '!=' 0 ']' 2026-02-17T12:19:58.783 INFO:tasks.workunit.client.0.trial128.stdout:TEST PASSED 2026-02-17T12:19:58.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1162: _scrub_snaps_multi: echo 'TEST PASSED' 2026-02-17T12:19:58.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1163: _scrub_snaps_multi: return 0 2026-02-17T12:19:58.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1171: TEST_scrub_snaps_replica: err=0 2026-02-17T12:19:58.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1172: TEST_scrub_snaps_replica: CEPH_ARGS='--fsid=20d75eec-1191-49ff-addc-3a8f083bd5b3 --auth-supported=none --mon-host=127.0.0.1:7121 ' 2026-02-17T12:19:58.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:1173: TEST_scrub_snaps_replica: return 0 2026-02-17T12:19:58.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:40: run: teardown td/osd-scrub-snaps 2026-02-17T12:19:58.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-snaps 2026-02-17T12:19:58.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:19:58.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-17T12:19:58.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:19:58.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:19:58.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:19:58.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:19:58.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:19:58.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:19:58.790 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:19:58.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:19:58.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:19:58.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:19:58.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:19:58.793 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:19:58.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:19:58.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:19:58.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:19:58.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:19:58.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:19:58.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:19:58.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-snaps 2026-02-17T12:19:58.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:19:58.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:58.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:19:58.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.452141 2026-02-17T12:19:58.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:19:58.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:19:58.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T12:19:58.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-scrub-snaps 0 2026-02-17T12:19:58.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-snaps 2026-02-17T12:19:58.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T12:19:58.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-snaps KILL 2026-02-17T12:19:58.812 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:19:58.813 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:19:58.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:19:58.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:19:58.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:19:58.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:19:58.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:19:58.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:19:58.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:19:58.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:19:58.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:19:58.818 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:19:58.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:19:58.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:19:58.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:19:58.819 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:19:58.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:19:58.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T12:19:58.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-snaps 2026-02-17T12:19:58.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:19:58.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:58.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.452141 2026-02-17T12:19:58.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.452141 2026-02-17T12:19:58.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:19:58.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:19:58.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T12:19:58.825 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T12:19:58.826 DEBUG:teuthology.orchestra.run.trial128:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T12:19:58.876 INFO:tasks.workunit:Running workunit scrub/osd-scrub-test.sh... 2026-02-17T12:19:58.876 DEBUG:teuthology.orchestra.run.trial128:workunit test scrub/osd-scrub-test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3c96dfb3f50b3d02598a82af7104764bfdcd3919 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh 2026-02-17T12:19:58.929 INFO:tasks.workunit.client.0.trial128.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T12:19:58.933 INFO:tasks.workunit.client.0.trial128.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T12:19:58.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T12:19:58.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T12:19:58.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T12:19:58.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T12:19:58.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T12:19:58.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T12:19:58.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T12:19:58.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T12:19:58.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-scrub-test 2026-02-17T12:19:58.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:21: run: local dir=td/osd-scrub-test 2026-02-17T12:19:58.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:22: run: shift 2026-02-17T12:19:58.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:24: run: export CEPH_MON=127.0.0.1:7138 2026-02-17T12:19:58.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:24: run: CEPH_MON=127.0.0.1:7138 2026-02-17T12:19:58.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:25: run: export CEPH_ARGS 2026-02-17T12:19:58.934 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:26: run: uuidgen 2026-02-17T12:19:58.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:26: run: CEPH_ARGS+='--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none ' 2026-02-17T12:19:58.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:27: run: CEPH_ARGS+='--mon-host=127.0.0.1:7138 ' 2026-02-17T12:19:58.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:29: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-17T12:19:58.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:30: run: set 2026-02-17T12:19:58.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:30: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T12:19:58.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:30: run: local 'funcs=TEST_deep_scrub_abort 2026-02-17T12:19:58.937 INFO:tasks.workunit.client.0.trial128.stderr:TEST_dump_scrub_schedule 2026-02-17T12:19:58.937 INFO:tasks.workunit.client.0.trial128.stderr:TEST_interval_changes 2026-02-17T12:19:58.937 INFO:tasks.workunit.client.0.trial128.stderr:TEST_just_deep_scrubs 2026-02-17T12:19:58.937 INFO:tasks.workunit.client.0.trial128.stderr:TEST_pg_dump_objects_scrubbed 2026-02-17T12:19:58.937 INFO:tasks.workunit.client.0.trial128.stderr:TEST_scrub_abort 2026-02-17T12:19:58.937 INFO:tasks.workunit.client.0.trial128.stderr:TEST_scrub_extended_sleep 2026-02-17T12:19:58.937 INFO:tasks.workunit.client.0.trial128.stderr:TEST_scrub_permit_time 2026-02-17T12:19:58.937 INFO:tasks.workunit.client.0.trial128.stderr:TEST_scrub_test' 2026-02-17T12:19:58.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T12:19:58.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_deep_scrub_abort -------------------' 2026-02-17T12:19:58.938 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Prepare Test TEST_deep_scrub_abort ------------------- 2026-02-17T12:19:58.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T12:19:58.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T12:19:58.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T12:19:58.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:19:58.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:19:58.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:19:58.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:19:58.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:19:58.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:19:58.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:19:58.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:19:58.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:19:58.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:19:58.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:19:58.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:19:58.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:19:58.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:19:58.942 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:19:58.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:19:58.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:19:58.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:19:58.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:19:58.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:19:58.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:19:58.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:19:58.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:19:58.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:58.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:19:58.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:19:58.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:19:58.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:19:58.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T12:19:58.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:19:58.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:58.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:19:58.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.508089 2026-02-17T12:19:58.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:19:58.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T12:19:58.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T12:19:58.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:19:58.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T12:19:58.954 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Run Test TEST_deep_scrub_abort ----------------------- 2026-02-17T12:19:58.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_deep_scrub_abort -----------------------' 2026-02-17T12:19:58.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_deep_scrub_abort td/osd-scrub-test 2026-02-17T12:19:58.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:414: TEST_deep_scrub_abort: local dir=td/osd-scrub-test 2026-02-17T12:19:58.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:415: TEST_deep_scrub_abort: _scrub_abort td/osd-scrub-test deep-scrub 2026-02-17T12:19:58.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:298: _scrub_abort: local dir=td/osd-scrub-test 2026-02-17T12:19:58.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:299: _scrub_abort: local poolname=test 2026-02-17T12:19:58.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:300: _scrub_abort: local OSDS=3 2026-02-17T12:19:58.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:301: _scrub_abort: local objects=1000 2026-02-17T12:19:58.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:302: _scrub_abort: local type=deep-scrub 2026-02-17T12:19:58.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:304: _scrub_abort: TESTDATA=testdata.508089 2026-02-17T12:19:58.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:305: _scrub_abort: test deep-scrub = scrub 2026-02-17T12:19:58.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:310: _scrub_abort: stopscrub=nodeep-scrub 2026-02-17T12:19:58.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:311: _scrub_abort: check=nodeep_scrub 2026-02-17T12:19:58.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:314: _scrub_abort: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-17T12:19:58.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-test 2026-02-17T12:19:58.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:19:58.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:19:58.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:19:58.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-test/a 2026-02-17T12:19:58.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-17T12:19:58.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:19:58.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:19:58.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:19:58.977 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:19:58.977 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:58.977 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:19:58.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:19:58.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-17T12:19:59.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:19:59.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:19:59.007 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:19:59.007 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:19:59.007 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:19:59.007 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:19:59.007 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:19:59.007 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:19:59.007 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:19:59.007 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:19:59.007 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:59.008 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:19:59.008 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-mon.a.asok 2026-02-17T12:19:59.008 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:19:59.008 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.508089/ceph-mon.a.asok config get fsid 2026-02-17T12:19:59.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:19:59.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:19:59.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:19:59.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:19:59.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:19:59.063 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:19:59.063 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:19:59.063 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:19:59.064 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:19:59.064 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:59.064 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:19:59.065 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-mon.a.asok 2026-02-17T12:19:59.065 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:19:59.065 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.508089/ceph-mon.a.asok config get mon_host 2026-02-17T12:19:59.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:315: _scrub_abort: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-17T12:19:59.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-test 2026-02-17T12:19:59.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:19:59.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:19:59.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:19:59.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-test/x 2026-02-17T12:19:59.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:19:59.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:19:59.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:19:59.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:19:59.213 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:19:59.213 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:59.213 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:19:59.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:19:59.214 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:19:59.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T12:19:59.232 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: expr 3 - 1 2026-02-17T12:19:59.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: seq 0 2 2026-02-17T12:19:59.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:19:59.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:322: _scrub_abort: run_osd td/osd-scrub-test 0 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T12:19:59.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:19:59.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:19:59.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:19:59.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:19:59.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-17T12:19:59.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:19:59.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:19:59.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:19:59.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:19:59.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-17T12:19:59.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-17T12:19:59.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:19:59.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:19:59.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:19:59.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:19:59.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:19:59.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:19:59.238 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:19:59.239 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:19:59.239 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:19:59.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:19:59.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:19:59.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:19:59.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:19:59.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:19:59.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:19:59.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:19:59.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:19:59.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:19:59.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:19:59.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:19:59.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:19:59.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-17T12:19:59.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-17T12:19:59.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:19:59.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2fce9522-e818-49ae-baff-fb60c4d69cb9 2026-02-17T12:19:59.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 2fce9522-e818-49ae-baff-fb60c4d69cb9' 2026-02-17T12:19:59.241 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 2fce9522-e818-49ae-baff-fb60c4d69cb9 2026-02-17T12:19:59.241 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:19:59.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDvXJRpPlNaDhAAdvgcE/LaJzm0NqVhhFxzTw== 2026-02-17T12:19:59.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDvXJRpPlNaDhAAdvgcE/LaJzm0NqVhhFxzTw=="}' 2026-02-17T12:19:59.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2fce9522-e818-49ae-baff-fb60c4d69cb9 -i td/osd-scrub-test/0/new.json 2026-02-17T12:19:59.336 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:19:59.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-17T12:19:59.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQDvXJRpPlNaDhAAdvgcE/LaJzm0NqVhhFxzTw== --osd-uuid 2fce9522-e818-49ae-baff-fb60c4d69cb9 2026-02-17T12:19:59.361 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:19:59.352+0000 7fedfaaf58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:59.362 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:19:59.352+0000 7fedfaaf58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:59.363 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:19:59.356+0000 7fedfaaf58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:19:59.364 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:19:59.356+0000 7fedfaaf58c0 -1 bdev(0x556042c69c00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:19:59.364 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:19:59.356+0000 7fedfaaf58c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T12:20:02.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-17T12:20:02.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:20:02.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:20:02.166 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:20:02.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:20:02.357 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:20:02.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:20:02.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T12:20:02.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:20:02.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:20:02.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:20:02.381 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:02.371+0000 7fdb169998c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:02.383 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:02.371+0000 7fdb169998c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:02.384 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:02.375+0000 7fdb169998c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:02.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:20:02.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:20:02.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:20:02.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:20:02.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:20:02.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:02.511 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:20:02.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:20:02.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:02.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:20:02.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:03.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:03.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:03.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:20:03.666 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:20:03.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:03.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:20:03.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:03.909 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:03.899+0000 7fdb169998c0 -1 Falling back to public interface 2026-02-17T12:20:04.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:04.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:04.818 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:20:04.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:20:04.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:04.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:20:04.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:05.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:05.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:05.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:20:05.974 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:20:05.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:05.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:20:06.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:06.931 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:06.919+0000 7fdb169998c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:20:07.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:07.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:07.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:20:07.131 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:20:07.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:07.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:20:07.283 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2813437670,v1:127.0.0.1:6803/2813437670] [v2:127.0.0.1:6804/2813437670,v1:127.0.0.1:6805/2813437670] exists,up 2fce9522-e818-49ae-baff-fb60c4d69cb9 2026-02-17T12:20:07.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:20:07.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:20:07.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:20:07.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:20:07.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:322: _scrub_abort: run_osd td/osd-scrub-test 1 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T12:20:07.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:20:07.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:20:07.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:20:07.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:20:07.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-17T12:20:07.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:20:07.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:20:07.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:20:07.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:20:07.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-17T12:20:07.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-17T12:20:07.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:20:07.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:20:07.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:20:07.289 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:20:07.289 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:20:07.289 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:20:07.289 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:20:07.289 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:20:07.289 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:20:07.289 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:20:07.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:20:07.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:20:07.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:20:07.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:20:07.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:20:07.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:20:07.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:20:07.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:20:07.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:20:07.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:20:07.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:20:07.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-17T12:20:07.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-17T12:20:07.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:20:07.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=9f6396de-cfd2-443f-bf8b-9e5c51daabc4 2026-02-17T12:20:07.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 9f6396de-cfd2-443f-bf8b-9e5c51daabc4' 2026-02-17T12:20:07.291 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 9f6396de-cfd2-443f-bf8b-9e5c51daabc4 2026-02-17T12:20:07.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:20:07.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQD3XJRpIJxsERAA5n77+FpOywWCS7Isuo6rqQ== 2026-02-17T12:20:07.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQD3XJRpIJxsERAA5n77+FpOywWCS7Isuo6rqQ=="}' 2026-02-17T12:20:07.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 9f6396de-cfd2-443f-bf8b-9e5c51daabc4 -i td/osd-scrub-test/1/new.json 2026-02-17T12:20:07.453 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:20:07.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-17T12:20:07.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQD3XJRpIJxsERAA5n77+FpOywWCS7Isuo6rqQ== --osd-uuid 9f6396de-cfd2-443f-bf8b-9e5c51daabc4 2026-02-17T12:20:07.480 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:07.471+0000 7f04473d28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:07.481 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:07.471+0000 7f04473d28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:07.482 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:07.471+0000 7f04473d28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:07.483 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:07.471+0000 7f04473d28c0 -1 bdev(0x56485e3a7c00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:20:07.483 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:07.471+0000 7f04473d28c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T12:20:09.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-17T12:20:09.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:20:09.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:20:09.751 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:20:09.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:20:09.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:20:09.948 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:20:09.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T12:20:09.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:20:09.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:20:09.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:20:09.964 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:09.955+0000 7f67e73908c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:09.966 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:09.955+0000 7f67e73908c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:09.967 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:09.955+0000 7f67e73908c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:10.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:20:10.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:20:10.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:20:10.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:20:10.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:20:10.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:10.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:20:10.109 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:20:10.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:10.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:20:10.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:11.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:11.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:11.266 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:20:11.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:20:11.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:11.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:20:11.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:11.481 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:11.471+0000 7f67e73908c0 -1 Falling back to public interface 2026-02-17T12:20:12.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:12.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:12.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:20:12.419 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:20:12.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:12.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:20:12.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:13.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:13.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:13.573 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:20:13.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:20:13.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:13.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:20:13.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:14.498 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:14.487+0000 7f67e73908c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:20:14.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:14.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:14.729 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:20:14.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:20:14.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:14.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:20:14.881 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4209619519,v1:127.0.0.1:6811/4209619519] [v2:127.0.0.1:6812/4209619519,v1:127.0.0.1:6813/4209619519] exists,up 9f6396de-cfd2-443f-bf8b-9e5c51daabc4 2026-02-17T12:20:14.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:20:14.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:20:14.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:20:14.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:20:14.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:322: _scrub_abort: run_osd td/osd-scrub-test 2 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T12:20:14.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:20:14.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:20:14.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:20:14.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:20:14.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-17T12:20:14.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:20:14.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:20:14.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:20:14.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:20:14.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-17T12:20:14.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-17T12:20:14.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:20:14.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:20:14.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:20:14.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:20:14.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:20:14.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:20:14.887 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:20:14.887 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:20:14.887 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:20:14.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:20:14.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:20:14.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:20:14.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:20:14.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:20:14.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:20:14.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:20:14.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:20:14.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:20:14.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:20:14.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:20:14.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:20:14.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-17T12:20:14.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-17T12:20:14.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:20:14.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=cc0d919b-419c-4dac-a9d0-924145f1c1df 2026-02-17T12:20:14.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 cc0d919b-419c-4dac-a9d0-924145f1c1df' 2026-02-17T12:20:14.889 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 cc0d919b-419c-4dac-a9d0-924145f1c1df 2026-02-17T12:20:14.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:20:14.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQD+XJRpuv4iNRAAO+vfjTD2Tcyai8kXetoZbQ== 2026-02-17T12:20:14.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQD+XJRpuv4iNRAAO+vfjTD2Tcyai8kXetoZbQ=="}' 2026-02-17T12:20:14.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new cc0d919b-419c-4dac-a9d0-924145f1c1df -i td/osd-scrub-test/2/new.json 2026-02-17T12:20:15.053 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:20:15.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-17T12:20:15.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQD+XJRpuv4iNRAAO+vfjTD2Tcyai8kXetoZbQ== --osd-uuid cc0d919b-419c-4dac-a9d0-924145f1c1df 2026-02-17T12:20:15.080 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:15.071+0000 7fed7a1618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:15.081 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:15.071+0000 7fed7a1618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:15.082 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:15.071+0000 7fed7a1618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:15.083 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:15.071+0000 7fed7a1618c0 -1 bdev(0x55dc4afe3c00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:20:15.083 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:15.071+0000 7fed7a1618c0 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-17T12:20:17.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-17T12:20:17.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:20:17.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:20:17.358 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T12:20:17.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:20:17.554 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T12:20:17.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:20:17.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T12:20:17.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:20:17.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:20:17.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:20:17.577 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:17.567+0000 7ff51138a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:17.579 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:17.567+0000 7ff51138a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:17.580 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:17.571+0000 7ff51138a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:20:17.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:20:17.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:20:17.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:20:17.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:20:17.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:20:17.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:17.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:20:17.705 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:20:17.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:17.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:20:17.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:18.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:18.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:18.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:20:18.859 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:20:18.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:18.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:20:19.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:19.325 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:19.315+0000 7ff51138a8c0 -1 Falling back to public interface 2026-02-17T12:20:20.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:20.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:20.014 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:20:20.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:20:20.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:20.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:20:20.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:21.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:21.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:21.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:20:21.169 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:20:21.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:21.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:20:21.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:22.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:22.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:22.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:20:22.325 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:20:22.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:22.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:20:22.427 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:20:22.419+0000 7ff51138a8c0 -1 osd.2 0 log_to_monitors true 2026-02-17T12:20:22.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:20:23.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:20:23.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:20:23.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:20:23.481 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:20:23.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:20:23.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:20:23.637 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1629400938,v1:127.0.0.1:6819/1629400938] [v2:127.0.0.1:6820/1629400938,v1:127.0.0.1:6821/1629400938] exists,up cc0d919b-419c-4dac-a9d0-924145f1c1df 2026-02-17T12:20:23.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:20:23.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:20:23.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:20:23.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: create_pool test 1 1 2026-02-17T12:20:23.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T12:20:23.837 INFO:tasks.workunit.client.0.trial128.stderr:pool 'test' created 2026-02-17T12:20:23.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:20:24.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:331: _scrub_abort: wait_for_clean 2026-02-17T12:20:24.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:20:24.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:20:24.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:20:24.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:20:24.853 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:20:24.853 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:20:24.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:20:24.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:20:24.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:20:24.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:20:24.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:20:24.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:20:24.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:20:24.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:20:24.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:20:25.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:20:25.118 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:20:25.118 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:20:25.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:20:25.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:25.119 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:20:25.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836499 2026-02-17T12:20:25.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836499 2026-02-17T12:20:25.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499' 2026-02-17T12:20:25.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:25.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:20:25.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705675 2026-02-17T12:20:25.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705675 2026-02-17T12:20:25.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-38654705675' 2026-02-17T12:20:25.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:25.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:20:25.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574851 2026-02-17T12:20:25.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574851 2026-02-17T12:20:25.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836499 1-38654705675 2-55834574851' 2026-02-17T12:20:25.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:25.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836499 2026-02-17T12:20:25.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:25.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:20:25.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836499 2026-02-17T12:20:25.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:25.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836499 2026-02-17T12:20:25.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2026-02-17T12:20:25.337 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836499 2026-02-17T12:20:25.338 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:20:25.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836499 2026-02-17T12:20:25.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:20:26.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:20:26.502 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:20:26.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836499 2026-02-17T12:20:26.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:26.653 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705675 2026-02-17T12:20:26.653 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:26.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:20:26.655 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705675 2026-02-17T12:20:26.655 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:26.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705675 2026-02-17T12:20:26.657 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705675 2026-02-17T12:20:26.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-17T12:20:26.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:20:26.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705675 2026-02-17T12:20:26.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:26.821 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574851 2026-02-17T12:20:26.821 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:26.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:20:26.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574851 2026-02-17T12:20:26.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:26.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574851 2026-02-17T12:20:26.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-17T12:20:26.825 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574851 2026-02-17T12:20:26.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:20:26.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-17T12:20:26.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:20:26.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:20:26.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:20:27.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:20:27.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:20:27.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:20:27.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:20:27.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:20:27.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:20:27.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:20:27.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:20:27.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:20:27.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:20:27.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:20:27.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:20:27.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:20:27.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:20:27.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:20:27.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: ceph osd dump 2026-02-17T12:20:27.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: grep '^pool.*['\'']test['\'']' 2026-02-17T12:20:27.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: awk '{ print $2 }' 2026-02-17T12:20:27.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: poolid=1 2026-02-17T12:20:27.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:334: _scrub_abort: dd if=/dev/urandom of=testdata.508089 bs=1032 count=1 2026-02-17T12:20:27.634 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records in 2026-02-17T12:20:27.635 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records out 2026-02-17T12:20:27.635 INFO:tasks.workunit.client.0.trial128.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.5055e-05 s, 29.4 MB/s 2026-02-17T12:20:27.635 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: seq 1 1000 2026-02-17T12:20:27.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:27.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj1 testdata.508089 2026-02-17T12:20:27.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:27.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj2 testdata.508089 2026-02-17T12:20:27.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:27.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj3 testdata.508089 2026-02-17T12:20:27.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:27.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj4 testdata.508089 2026-02-17T12:20:27.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:27.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj5 testdata.508089 2026-02-17T12:20:27.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:27.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj6 testdata.508089 2026-02-17T12:20:27.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:27.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj7 testdata.508089 2026-02-17T12:20:27.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:27.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj8 testdata.508089 2026-02-17T12:20:27.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:27.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj9 testdata.508089 2026-02-17T12:20:27.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:27.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj10 testdata.508089 2026-02-17T12:20:27.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:27.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj11 testdata.508089 2026-02-17T12:20:27.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:27.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj12 testdata.508089 2026-02-17T12:20:27.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:27.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj13 testdata.508089 2026-02-17T12:20:27.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:27.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj14 testdata.508089 2026-02-17T12:20:27.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:27.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj15 testdata.508089 2026-02-17T12:20:27.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:27.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj16 testdata.508089 2026-02-17T12:20:28.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj17 testdata.508089 2026-02-17T12:20:28.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj18 testdata.508089 2026-02-17T12:20:28.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj19 testdata.508089 2026-02-17T12:20:28.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj20 testdata.508089 2026-02-17T12:20:28.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj21 testdata.508089 2026-02-17T12:20:28.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj22 testdata.508089 2026-02-17T12:20:28.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj23 testdata.508089 2026-02-17T12:20:28.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj24 testdata.508089 2026-02-17T12:20:28.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj25 testdata.508089 2026-02-17T12:20:28.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj26 testdata.508089 2026-02-17T12:20:28.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj27 testdata.508089 2026-02-17T12:20:28.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj28 testdata.508089 2026-02-17T12:20:28.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj29 testdata.508089 2026-02-17T12:20:28.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj30 testdata.508089 2026-02-17T12:20:28.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj31 testdata.508089 2026-02-17T12:20:28.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj32 testdata.508089 2026-02-17T12:20:28.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj33 testdata.508089 2026-02-17T12:20:28.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj34 testdata.508089 2026-02-17T12:20:28.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj35 testdata.508089 2026-02-17T12:20:28.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj36 testdata.508089 2026-02-17T12:20:28.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj37 testdata.508089 2026-02-17T12:20:28.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj38 testdata.508089 2026-02-17T12:20:28.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj39 testdata.508089 2026-02-17T12:20:28.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj40 testdata.508089 2026-02-17T12:20:28.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj41 testdata.508089 2026-02-17T12:20:28.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj42 testdata.508089 2026-02-17T12:20:28.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj43 testdata.508089 2026-02-17T12:20:28.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj44 testdata.508089 2026-02-17T12:20:28.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj45 testdata.508089 2026-02-17T12:20:28.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj46 testdata.508089 2026-02-17T12:20:28.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj47 testdata.508089 2026-02-17T12:20:28.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj48 testdata.508089 2026-02-17T12:20:28.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj49 testdata.508089 2026-02-17T12:20:28.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj50 testdata.508089 2026-02-17T12:20:28.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj51 testdata.508089 2026-02-17T12:20:28.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj52 testdata.508089 2026-02-17T12:20:28.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj53 testdata.508089 2026-02-17T12:20:28.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj54 testdata.508089 2026-02-17T12:20:28.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj55 testdata.508089 2026-02-17T12:20:28.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj56 testdata.508089 2026-02-17T12:20:28.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj57 testdata.508089 2026-02-17T12:20:28.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:28.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj58 testdata.508089 2026-02-17T12:20:29.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj59 testdata.508089 2026-02-17T12:20:29.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj60 testdata.508089 2026-02-17T12:20:29.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj61 testdata.508089 2026-02-17T12:20:29.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj62 testdata.508089 2026-02-17T12:20:29.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj63 testdata.508089 2026-02-17T12:20:29.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj64 testdata.508089 2026-02-17T12:20:29.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj65 testdata.508089 2026-02-17T12:20:29.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj66 testdata.508089 2026-02-17T12:20:29.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj67 testdata.508089 2026-02-17T12:20:29.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj68 testdata.508089 2026-02-17T12:20:29.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj69 testdata.508089 2026-02-17T12:20:29.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj70 testdata.508089 2026-02-17T12:20:29.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj71 testdata.508089 2026-02-17T12:20:29.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj72 testdata.508089 2026-02-17T12:20:29.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj73 testdata.508089 2026-02-17T12:20:29.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj74 testdata.508089 2026-02-17T12:20:29.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj75 testdata.508089 2026-02-17T12:20:29.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj76 testdata.508089 2026-02-17T12:20:29.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj77 testdata.508089 2026-02-17T12:20:29.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj78 testdata.508089 2026-02-17T12:20:29.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj79 testdata.508089 2026-02-17T12:20:29.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj80 testdata.508089 2026-02-17T12:20:29.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj81 testdata.508089 2026-02-17T12:20:29.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj82 testdata.508089 2026-02-17T12:20:29.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj83 testdata.508089 2026-02-17T12:20:29.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj84 testdata.508089 2026-02-17T12:20:29.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj85 testdata.508089 2026-02-17T12:20:29.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj86 testdata.508089 2026-02-17T12:20:29.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj87 testdata.508089 2026-02-17T12:20:29.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj88 testdata.508089 2026-02-17T12:20:29.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj89 testdata.508089 2026-02-17T12:20:29.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj90 testdata.508089 2026-02-17T12:20:29.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj91 testdata.508089 2026-02-17T12:20:29.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj92 testdata.508089 2026-02-17T12:20:29.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj93 testdata.508089 2026-02-17T12:20:29.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj94 testdata.508089 2026-02-17T12:20:29.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj95 testdata.508089 2026-02-17T12:20:29.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj96 testdata.508089 2026-02-17T12:20:29.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj97 testdata.508089 2026-02-17T12:20:29.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj98 testdata.508089 2026-02-17T12:20:29.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj99 testdata.508089 2026-02-17T12:20:29.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj100 testdata.508089 2026-02-17T12:20:29.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:29.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj101 testdata.508089 2026-02-17T12:20:30.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj102 testdata.508089 2026-02-17T12:20:30.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj103 testdata.508089 2026-02-17T12:20:30.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj104 testdata.508089 2026-02-17T12:20:30.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj105 testdata.508089 2026-02-17T12:20:30.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj106 testdata.508089 2026-02-17T12:20:30.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj107 testdata.508089 2026-02-17T12:20:30.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj108 testdata.508089 2026-02-17T12:20:30.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj109 testdata.508089 2026-02-17T12:20:30.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj110 testdata.508089 2026-02-17T12:20:30.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj111 testdata.508089 2026-02-17T12:20:30.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj112 testdata.508089 2026-02-17T12:20:30.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj113 testdata.508089 2026-02-17T12:20:30.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj114 testdata.508089 2026-02-17T12:20:30.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj115 testdata.508089 2026-02-17T12:20:30.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj116 testdata.508089 2026-02-17T12:20:30.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj117 testdata.508089 2026-02-17T12:20:30.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj118 testdata.508089 2026-02-17T12:20:30.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj119 testdata.508089 2026-02-17T12:20:30.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj120 testdata.508089 2026-02-17T12:20:30.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj121 testdata.508089 2026-02-17T12:20:30.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj122 testdata.508089 2026-02-17T12:20:30.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj123 testdata.508089 2026-02-17T12:20:30.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj124 testdata.508089 2026-02-17T12:20:30.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj125 testdata.508089 2026-02-17T12:20:30.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj126 testdata.508089 2026-02-17T12:20:30.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj127 testdata.508089 2026-02-17T12:20:30.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj128 testdata.508089 2026-02-17T12:20:30.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj129 testdata.508089 2026-02-17T12:20:30.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj130 testdata.508089 2026-02-17T12:20:30.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj131 testdata.508089 2026-02-17T12:20:30.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj132 testdata.508089 2026-02-17T12:20:30.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj133 testdata.508089 2026-02-17T12:20:30.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj134 testdata.508089 2026-02-17T12:20:30.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj135 testdata.508089 2026-02-17T12:20:30.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj136 testdata.508089 2026-02-17T12:20:30.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj137 testdata.508089 2026-02-17T12:20:30.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj138 testdata.508089 2026-02-17T12:20:30.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj139 testdata.508089 2026-02-17T12:20:30.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj140 testdata.508089 2026-02-17T12:20:30.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj141 testdata.508089 2026-02-17T12:20:30.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj142 testdata.508089 2026-02-17T12:20:30.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj143 testdata.508089 2026-02-17T12:20:30.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:30.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj144 testdata.508089 2026-02-17T12:20:31.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj145 testdata.508089 2026-02-17T12:20:31.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj146 testdata.508089 2026-02-17T12:20:31.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj147 testdata.508089 2026-02-17T12:20:31.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj148 testdata.508089 2026-02-17T12:20:31.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj149 testdata.508089 2026-02-17T12:20:31.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj150 testdata.508089 2026-02-17T12:20:31.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj151 testdata.508089 2026-02-17T12:20:31.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj152 testdata.508089 2026-02-17T12:20:31.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj153 testdata.508089 2026-02-17T12:20:31.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj154 testdata.508089 2026-02-17T12:20:31.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj155 testdata.508089 2026-02-17T12:20:31.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj156 testdata.508089 2026-02-17T12:20:31.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj157 testdata.508089 2026-02-17T12:20:31.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj158 testdata.508089 2026-02-17T12:20:31.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj159 testdata.508089 2026-02-17T12:20:31.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj160 testdata.508089 2026-02-17T12:20:31.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj161 testdata.508089 2026-02-17T12:20:31.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj162 testdata.508089 2026-02-17T12:20:31.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj163 testdata.508089 2026-02-17T12:20:31.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj164 testdata.508089 2026-02-17T12:20:31.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj165 testdata.508089 2026-02-17T12:20:31.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj166 testdata.508089 2026-02-17T12:20:31.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj167 testdata.508089 2026-02-17T12:20:31.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj168 testdata.508089 2026-02-17T12:20:31.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj169 testdata.508089 2026-02-17T12:20:31.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj170 testdata.508089 2026-02-17T12:20:31.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj171 testdata.508089 2026-02-17T12:20:31.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj172 testdata.508089 2026-02-17T12:20:31.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj173 testdata.508089 2026-02-17T12:20:31.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj174 testdata.508089 2026-02-17T12:20:31.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj175 testdata.508089 2026-02-17T12:20:31.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj176 testdata.508089 2026-02-17T12:20:31.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj177 testdata.508089 2026-02-17T12:20:31.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj178 testdata.508089 2026-02-17T12:20:31.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj179 testdata.508089 2026-02-17T12:20:31.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj180 testdata.508089 2026-02-17T12:20:31.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj181 testdata.508089 2026-02-17T12:20:31.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj182 testdata.508089 2026-02-17T12:20:31.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj183 testdata.508089 2026-02-17T12:20:31.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj184 testdata.508089 2026-02-17T12:20:31.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj185 testdata.508089 2026-02-17T12:20:31.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:31.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj186 testdata.508089 2026-02-17T12:20:32.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj187 testdata.508089 2026-02-17T12:20:32.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj188 testdata.508089 2026-02-17T12:20:32.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj189 testdata.508089 2026-02-17T12:20:32.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj190 testdata.508089 2026-02-17T12:20:32.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj191 testdata.508089 2026-02-17T12:20:32.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj192 testdata.508089 2026-02-17T12:20:32.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj193 testdata.508089 2026-02-17T12:20:32.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj194 testdata.508089 2026-02-17T12:20:32.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj195 testdata.508089 2026-02-17T12:20:32.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj196 testdata.508089 2026-02-17T12:20:32.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj197 testdata.508089 2026-02-17T12:20:32.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj198 testdata.508089 2026-02-17T12:20:32.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj199 testdata.508089 2026-02-17T12:20:32.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj200 testdata.508089 2026-02-17T12:20:32.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj201 testdata.508089 2026-02-17T12:20:32.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj202 testdata.508089 2026-02-17T12:20:32.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj203 testdata.508089 2026-02-17T12:20:32.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj204 testdata.508089 2026-02-17T12:20:32.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj205 testdata.508089 2026-02-17T12:20:32.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj206 testdata.508089 2026-02-17T12:20:32.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj207 testdata.508089 2026-02-17T12:20:32.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj208 testdata.508089 2026-02-17T12:20:32.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj209 testdata.508089 2026-02-17T12:20:32.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj210 testdata.508089 2026-02-17T12:20:32.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj211 testdata.508089 2026-02-17T12:20:32.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj212 testdata.508089 2026-02-17T12:20:32.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj213 testdata.508089 2026-02-17T12:20:32.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj214 testdata.508089 2026-02-17T12:20:32.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj215 testdata.508089 2026-02-17T12:20:32.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj216 testdata.508089 2026-02-17T12:20:32.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj217 testdata.508089 2026-02-17T12:20:32.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj218 testdata.508089 2026-02-17T12:20:32.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj219 testdata.508089 2026-02-17T12:20:32.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj220 testdata.508089 2026-02-17T12:20:32.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj221 testdata.508089 2026-02-17T12:20:32.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj222 testdata.508089 2026-02-17T12:20:32.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj223 testdata.508089 2026-02-17T12:20:32.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj224 testdata.508089 2026-02-17T12:20:32.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj225 testdata.508089 2026-02-17T12:20:32.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj226 testdata.508089 2026-02-17T12:20:32.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj227 testdata.508089 2026-02-17T12:20:32.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj228 testdata.508089 2026-02-17T12:20:32.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj229 testdata.508089 2026-02-17T12:20:32.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:32.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj230 testdata.508089 2026-02-17T12:20:33.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj231 testdata.508089 2026-02-17T12:20:33.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj232 testdata.508089 2026-02-17T12:20:33.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj233 testdata.508089 2026-02-17T12:20:33.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj234 testdata.508089 2026-02-17T12:20:33.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj235 testdata.508089 2026-02-17T12:20:33.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj236 testdata.508089 2026-02-17T12:20:33.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj237 testdata.508089 2026-02-17T12:20:33.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj238 testdata.508089 2026-02-17T12:20:33.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj239 testdata.508089 2026-02-17T12:20:33.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj240 testdata.508089 2026-02-17T12:20:33.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj241 testdata.508089 2026-02-17T12:20:33.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj242 testdata.508089 2026-02-17T12:20:33.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj243 testdata.508089 2026-02-17T12:20:33.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj244 testdata.508089 2026-02-17T12:20:33.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj245 testdata.508089 2026-02-17T12:20:33.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj246 testdata.508089 2026-02-17T12:20:33.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj247 testdata.508089 2026-02-17T12:20:33.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj248 testdata.508089 2026-02-17T12:20:33.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj249 testdata.508089 2026-02-17T12:20:33.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj250 testdata.508089 2026-02-17T12:20:33.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj251 testdata.508089 2026-02-17T12:20:33.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj252 testdata.508089 2026-02-17T12:20:33.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj253 testdata.508089 2026-02-17T12:20:33.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj254 testdata.508089 2026-02-17T12:20:33.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj255 testdata.508089 2026-02-17T12:20:33.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj256 testdata.508089 2026-02-17T12:20:33.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj257 testdata.508089 2026-02-17T12:20:33.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj258 testdata.508089 2026-02-17T12:20:33.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj259 testdata.508089 2026-02-17T12:20:33.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj260 testdata.508089 2026-02-17T12:20:33.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj261 testdata.508089 2026-02-17T12:20:33.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj262 testdata.508089 2026-02-17T12:20:33.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj263 testdata.508089 2026-02-17T12:20:33.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj264 testdata.508089 2026-02-17T12:20:33.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj265 testdata.508089 2026-02-17T12:20:33.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj266 testdata.508089 2026-02-17T12:20:33.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj267 testdata.508089 2026-02-17T12:20:33.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj268 testdata.508089 2026-02-17T12:20:33.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj269 testdata.508089 2026-02-17T12:20:33.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj270 testdata.508089 2026-02-17T12:20:33.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj271 testdata.508089 2026-02-17T12:20:33.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj272 testdata.508089 2026-02-17T12:20:33.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:33.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj273 testdata.508089 2026-02-17T12:20:34.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj274 testdata.508089 2026-02-17T12:20:34.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj275 testdata.508089 2026-02-17T12:20:34.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj276 testdata.508089 2026-02-17T12:20:34.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj277 testdata.508089 2026-02-17T12:20:34.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj278 testdata.508089 2026-02-17T12:20:34.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj279 testdata.508089 2026-02-17T12:20:34.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj280 testdata.508089 2026-02-17T12:20:34.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj281 testdata.508089 2026-02-17T12:20:34.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj282 testdata.508089 2026-02-17T12:20:34.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj283 testdata.508089 2026-02-17T12:20:34.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj284 testdata.508089 2026-02-17T12:20:34.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj285 testdata.508089 2026-02-17T12:20:34.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj286 testdata.508089 2026-02-17T12:20:34.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj287 testdata.508089 2026-02-17T12:20:34.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj288 testdata.508089 2026-02-17T12:20:34.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj289 testdata.508089 2026-02-17T12:20:34.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj290 testdata.508089 2026-02-17T12:20:34.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj291 testdata.508089 2026-02-17T12:20:34.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj292 testdata.508089 2026-02-17T12:20:34.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj293 testdata.508089 2026-02-17T12:20:34.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj294 testdata.508089 2026-02-17T12:20:34.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj295 testdata.508089 2026-02-17T12:20:34.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj296 testdata.508089 2026-02-17T12:20:34.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj297 testdata.508089 2026-02-17T12:20:34.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj298 testdata.508089 2026-02-17T12:20:34.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj299 testdata.508089 2026-02-17T12:20:34.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj300 testdata.508089 2026-02-17T12:20:34.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj301 testdata.508089 2026-02-17T12:20:34.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj302 testdata.508089 2026-02-17T12:20:34.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj303 testdata.508089 2026-02-17T12:20:34.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj304 testdata.508089 2026-02-17T12:20:34.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj305 testdata.508089 2026-02-17T12:20:34.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj306 testdata.508089 2026-02-17T12:20:34.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj307 testdata.508089 2026-02-17T12:20:34.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj308 testdata.508089 2026-02-17T12:20:34.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj309 testdata.508089 2026-02-17T12:20:34.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj310 testdata.508089 2026-02-17T12:20:34.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj311 testdata.508089 2026-02-17T12:20:34.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj312 testdata.508089 2026-02-17T12:20:34.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj313 testdata.508089 2026-02-17T12:20:34.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj314 testdata.508089 2026-02-17T12:20:34.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj315 testdata.508089 2026-02-17T12:20:34.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj316 testdata.508089 2026-02-17T12:20:34.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:34.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj317 testdata.508089 2026-02-17T12:20:35.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj318 testdata.508089 2026-02-17T12:20:35.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj319 testdata.508089 2026-02-17T12:20:35.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj320 testdata.508089 2026-02-17T12:20:35.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj321 testdata.508089 2026-02-17T12:20:35.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj322 testdata.508089 2026-02-17T12:20:35.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj323 testdata.508089 2026-02-17T12:20:35.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj324 testdata.508089 2026-02-17T12:20:35.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj325 testdata.508089 2026-02-17T12:20:35.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj326 testdata.508089 2026-02-17T12:20:35.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj327 testdata.508089 2026-02-17T12:20:35.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj328 testdata.508089 2026-02-17T12:20:35.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj329 testdata.508089 2026-02-17T12:20:35.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj330 testdata.508089 2026-02-17T12:20:35.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj331 testdata.508089 2026-02-17T12:20:35.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj332 testdata.508089 2026-02-17T12:20:35.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj333 testdata.508089 2026-02-17T12:20:35.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj334 testdata.508089 2026-02-17T12:20:35.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj335 testdata.508089 2026-02-17T12:20:35.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj336 testdata.508089 2026-02-17T12:20:35.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj337 testdata.508089 2026-02-17T12:20:35.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj338 testdata.508089 2026-02-17T12:20:35.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj339 testdata.508089 2026-02-17T12:20:35.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj340 testdata.508089 2026-02-17T12:20:35.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj341 testdata.508089 2026-02-17T12:20:35.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj342 testdata.508089 2026-02-17T12:20:35.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj343 testdata.508089 2026-02-17T12:20:35.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj344 testdata.508089 2026-02-17T12:20:35.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj345 testdata.508089 2026-02-17T12:20:35.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj346 testdata.508089 2026-02-17T12:20:35.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj347 testdata.508089 2026-02-17T12:20:35.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj348 testdata.508089 2026-02-17T12:20:35.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj349 testdata.508089 2026-02-17T12:20:35.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj350 testdata.508089 2026-02-17T12:20:35.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj351 testdata.508089 2026-02-17T12:20:35.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj352 testdata.508089 2026-02-17T12:20:35.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj353 testdata.508089 2026-02-17T12:20:35.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj354 testdata.508089 2026-02-17T12:20:35.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj355 testdata.508089 2026-02-17T12:20:35.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj356 testdata.508089 2026-02-17T12:20:35.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj357 testdata.508089 2026-02-17T12:20:35.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj358 testdata.508089 2026-02-17T12:20:35.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj359 testdata.508089 2026-02-17T12:20:35.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:35.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj360 testdata.508089 2026-02-17T12:20:36.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj361 testdata.508089 2026-02-17T12:20:36.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj362 testdata.508089 2026-02-17T12:20:36.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj363 testdata.508089 2026-02-17T12:20:36.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj364 testdata.508089 2026-02-17T12:20:36.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj365 testdata.508089 2026-02-17T12:20:36.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj366 testdata.508089 2026-02-17T12:20:36.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj367 testdata.508089 2026-02-17T12:20:36.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj368 testdata.508089 2026-02-17T12:20:36.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj369 testdata.508089 2026-02-17T12:20:36.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj370 testdata.508089 2026-02-17T12:20:36.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj371 testdata.508089 2026-02-17T12:20:36.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj372 testdata.508089 2026-02-17T12:20:36.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj373 testdata.508089 2026-02-17T12:20:36.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj374 testdata.508089 2026-02-17T12:20:36.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj375 testdata.508089 2026-02-17T12:20:36.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj376 testdata.508089 2026-02-17T12:20:36.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj377 testdata.508089 2026-02-17T12:20:36.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj378 testdata.508089 2026-02-17T12:20:36.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj379 testdata.508089 2026-02-17T12:20:36.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj380 testdata.508089 2026-02-17T12:20:36.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj381 testdata.508089 2026-02-17T12:20:36.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj382 testdata.508089 2026-02-17T12:20:36.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj383 testdata.508089 2026-02-17T12:20:36.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj384 testdata.508089 2026-02-17T12:20:36.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj385 testdata.508089 2026-02-17T12:20:36.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj386 testdata.508089 2026-02-17T12:20:36.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj387 testdata.508089 2026-02-17T12:20:36.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj388 testdata.508089 2026-02-17T12:20:36.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj389 testdata.508089 2026-02-17T12:20:36.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj390 testdata.508089 2026-02-17T12:20:36.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj391 testdata.508089 2026-02-17T12:20:36.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj392 testdata.508089 2026-02-17T12:20:36.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj393 testdata.508089 2026-02-17T12:20:36.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj394 testdata.508089 2026-02-17T12:20:36.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj395 testdata.508089 2026-02-17T12:20:36.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj396 testdata.508089 2026-02-17T12:20:36.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj397 testdata.508089 2026-02-17T12:20:36.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj398 testdata.508089 2026-02-17T12:20:36.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj399 testdata.508089 2026-02-17T12:20:36.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj400 testdata.508089 2026-02-17T12:20:36.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj401 testdata.508089 2026-02-17T12:20:36.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj402 testdata.508089 2026-02-17T12:20:36.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj403 testdata.508089 2026-02-17T12:20:36.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj404 testdata.508089 2026-02-17T12:20:36.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:36.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj405 testdata.508089 2026-02-17T12:20:37.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj406 testdata.508089 2026-02-17T12:20:37.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj407 testdata.508089 2026-02-17T12:20:37.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj408 testdata.508089 2026-02-17T12:20:37.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj409 testdata.508089 2026-02-17T12:20:37.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj410 testdata.508089 2026-02-17T12:20:37.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj411 testdata.508089 2026-02-17T12:20:37.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj412 testdata.508089 2026-02-17T12:20:37.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj413 testdata.508089 2026-02-17T12:20:37.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj414 testdata.508089 2026-02-17T12:20:37.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj415 testdata.508089 2026-02-17T12:20:37.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj416 testdata.508089 2026-02-17T12:20:37.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj417 testdata.508089 2026-02-17T12:20:37.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj418 testdata.508089 2026-02-17T12:20:37.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj419 testdata.508089 2026-02-17T12:20:37.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj420 testdata.508089 2026-02-17T12:20:37.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj421 testdata.508089 2026-02-17T12:20:37.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj422 testdata.508089 2026-02-17T12:20:37.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj423 testdata.508089 2026-02-17T12:20:37.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj424 testdata.508089 2026-02-17T12:20:37.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj425 testdata.508089 2026-02-17T12:20:37.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj426 testdata.508089 2026-02-17T12:20:37.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj427 testdata.508089 2026-02-17T12:20:37.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj428 testdata.508089 2026-02-17T12:20:37.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj429 testdata.508089 2026-02-17T12:20:37.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj430 testdata.508089 2026-02-17T12:20:37.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj431 testdata.508089 2026-02-17T12:20:37.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj432 testdata.508089 2026-02-17T12:20:37.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj433 testdata.508089 2026-02-17T12:20:37.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj434 testdata.508089 2026-02-17T12:20:37.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj435 testdata.508089 2026-02-17T12:20:37.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj436 testdata.508089 2026-02-17T12:20:37.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj437 testdata.508089 2026-02-17T12:20:37.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj438 testdata.508089 2026-02-17T12:20:37.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj439 testdata.508089 2026-02-17T12:20:37.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj440 testdata.508089 2026-02-17T12:20:37.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj441 testdata.508089 2026-02-17T12:20:37.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj442 testdata.508089 2026-02-17T12:20:37.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj443 testdata.508089 2026-02-17T12:20:37.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj444 testdata.508089 2026-02-17T12:20:37.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj445 testdata.508089 2026-02-17T12:20:37.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj446 testdata.508089 2026-02-17T12:20:37.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj447 testdata.508089 2026-02-17T12:20:37.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj448 testdata.508089 2026-02-17T12:20:37.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:37.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj449 testdata.508089 2026-02-17T12:20:38.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj450 testdata.508089 2026-02-17T12:20:38.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj451 testdata.508089 2026-02-17T12:20:38.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj452 testdata.508089 2026-02-17T12:20:38.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj453 testdata.508089 2026-02-17T12:20:38.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj454 testdata.508089 2026-02-17T12:20:38.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj455 testdata.508089 2026-02-17T12:20:38.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj456 testdata.508089 2026-02-17T12:20:38.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj457 testdata.508089 2026-02-17T12:20:38.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj458 testdata.508089 2026-02-17T12:20:38.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj459 testdata.508089 2026-02-17T12:20:38.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj460 testdata.508089 2026-02-17T12:20:38.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj461 testdata.508089 2026-02-17T12:20:38.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj462 testdata.508089 2026-02-17T12:20:38.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj463 testdata.508089 2026-02-17T12:20:38.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj464 testdata.508089 2026-02-17T12:20:38.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj465 testdata.508089 2026-02-17T12:20:38.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj466 testdata.508089 2026-02-17T12:20:38.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj467 testdata.508089 2026-02-17T12:20:38.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj468 testdata.508089 2026-02-17T12:20:38.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj469 testdata.508089 2026-02-17T12:20:38.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj470 testdata.508089 2026-02-17T12:20:38.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj471 testdata.508089 2026-02-17T12:20:38.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj472 testdata.508089 2026-02-17T12:20:38.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj473 testdata.508089 2026-02-17T12:20:38.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj474 testdata.508089 2026-02-17T12:20:38.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj475 testdata.508089 2026-02-17T12:20:38.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj476 testdata.508089 2026-02-17T12:20:38.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj477 testdata.508089 2026-02-17T12:20:38.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj478 testdata.508089 2026-02-17T12:20:38.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj479 testdata.508089 2026-02-17T12:20:38.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj480 testdata.508089 2026-02-17T12:20:38.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj481 testdata.508089 2026-02-17T12:20:38.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj482 testdata.508089 2026-02-17T12:20:38.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj483 testdata.508089 2026-02-17T12:20:38.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj484 testdata.508089 2026-02-17T12:20:38.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj485 testdata.508089 2026-02-17T12:20:38.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj486 testdata.508089 2026-02-17T12:20:38.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj487 testdata.508089 2026-02-17T12:20:38.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj488 testdata.508089 2026-02-17T12:20:38.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj489 testdata.508089 2026-02-17T12:20:38.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj490 testdata.508089 2026-02-17T12:20:38.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj491 testdata.508089 2026-02-17T12:20:38.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj492 testdata.508089 2026-02-17T12:20:38.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:38.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj493 testdata.508089 2026-02-17T12:20:39.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj494 testdata.508089 2026-02-17T12:20:39.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj495 testdata.508089 2026-02-17T12:20:39.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj496 testdata.508089 2026-02-17T12:20:39.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj497 testdata.508089 2026-02-17T12:20:39.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj498 testdata.508089 2026-02-17T12:20:39.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj499 testdata.508089 2026-02-17T12:20:39.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj500 testdata.508089 2026-02-17T12:20:39.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj501 testdata.508089 2026-02-17T12:20:39.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj502 testdata.508089 2026-02-17T12:20:39.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj503 testdata.508089 2026-02-17T12:20:39.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj504 testdata.508089 2026-02-17T12:20:39.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj505 testdata.508089 2026-02-17T12:20:39.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj506 testdata.508089 2026-02-17T12:20:39.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj507 testdata.508089 2026-02-17T12:20:39.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj508 testdata.508089 2026-02-17T12:20:39.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj509 testdata.508089 2026-02-17T12:20:39.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj510 testdata.508089 2026-02-17T12:20:39.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj511 testdata.508089 2026-02-17T12:20:39.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj512 testdata.508089 2026-02-17T12:20:39.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj513 testdata.508089 2026-02-17T12:20:39.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj514 testdata.508089 2026-02-17T12:20:39.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj515 testdata.508089 2026-02-17T12:20:39.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj516 testdata.508089 2026-02-17T12:20:39.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj517 testdata.508089 2026-02-17T12:20:39.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj518 testdata.508089 2026-02-17T12:20:39.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj519 testdata.508089 2026-02-17T12:20:39.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj520 testdata.508089 2026-02-17T12:20:39.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj521 testdata.508089 2026-02-17T12:20:39.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj522 testdata.508089 2026-02-17T12:20:39.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj523 testdata.508089 2026-02-17T12:20:39.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj524 testdata.508089 2026-02-17T12:20:39.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj525 testdata.508089 2026-02-17T12:20:39.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj526 testdata.508089 2026-02-17T12:20:39.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj527 testdata.508089 2026-02-17T12:20:39.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj528 testdata.508089 2026-02-17T12:20:39.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj529 testdata.508089 2026-02-17T12:20:39.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj530 testdata.508089 2026-02-17T12:20:39.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj531 testdata.508089 2026-02-17T12:20:39.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj532 testdata.508089 2026-02-17T12:20:39.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj533 testdata.508089 2026-02-17T12:20:39.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj534 testdata.508089 2026-02-17T12:20:39.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj535 testdata.508089 2026-02-17T12:20:39.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:39.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj536 testdata.508089 2026-02-17T12:20:40.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj537 testdata.508089 2026-02-17T12:20:40.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj538 testdata.508089 2026-02-17T12:20:40.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj539 testdata.508089 2026-02-17T12:20:40.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj540 testdata.508089 2026-02-17T12:20:40.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj541 testdata.508089 2026-02-17T12:20:40.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj542 testdata.508089 2026-02-17T12:20:40.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj543 testdata.508089 2026-02-17T12:20:40.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj544 testdata.508089 2026-02-17T12:20:40.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj545 testdata.508089 2026-02-17T12:20:40.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj546 testdata.508089 2026-02-17T12:20:40.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj547 testdata.508089 2026-02-17T12:20:40.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj548 testdata.508089 2026-02-17T12:20:40.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj549 testdata.508089 2026-02-17T12:20:40.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj550 testdata.508089 2026-02-17T12:20:40.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj551 testdata.508089 2026-02-17T12:20:40.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj552 testdata.508089 2026-02-17T12:20:40.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj553 testdata.508089 2026-02-17T12:20:40.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj554 testdata.508089 2026-02-17T12:20:40.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj555 testdata.508089 2026-02-17T12:20:40.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj556 testdata.508089 2026-02-17T12:20:40.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj557 testdata.508089 2026-02-17T12:20:40.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj558 testdata.508089 2026-02-17T12:20:40.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj559 testdata.508089 2026-02-17T12:20:40.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj560 testdata.508089 2026-02-17T12:20:40.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj561 testdata.508089 2026-02-17T12:20:40.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj562 testdata.508089 2026-02-17T12:20:40.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj563 testdata.508089 2026-02-17T12:20:40.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj564 testdata.508089 2026-02-17T12:20:40.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj565 testdata.508089 2026-02-17T12:20:40.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj566 testdata.508089 2026-02-17T12:20:40.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj567 testdata.508089 2026-02-17T12:20:40.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj568 testdata.508089 2026-02-17T12:20:40.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj569 testdata.508089 2026-02-17T12:20:40.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj570 testdata.508089 2026-02-17T12:20:40.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj571 testdata.508089 2026-02-17T12:20:40.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj572 testdata.508089 2026-02-17T12:20:40.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj573 testdata.508089 2026-02-17T12:20:40.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj574 testdata.508089 2026-02-17T12:20:40.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj575 testdata.508089 2026-02-17T12:20:40.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj576 testdata.508089 2026-02-17T12:20:40.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj577 testdata.508089 2026-02-17T12:20:40.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj578 testdata.508089 2026-02-17T12:20:40.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj579 testdata.508089 2026-02-17T12:20:40.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:40.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj580 testdata.508089 2026-02-17T12:20:41.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj581 testdata.508089 2026-02-17T12:20:41.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj582 testdata.508089 2026-02-17T12:20:41.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj583 testdata.508089 2026-02-17T12:20:41.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj584 testdata.508089 2026-02-17T12:20:41.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj585 testdata.508089 2026-02-17T12:20:41.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj586 testdata.508089 2026-02-17T12:20:41.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj587 testdata.508089 2026-02-17T12:20:41.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj588 testdata.508089 2026-02-17T12:20:41.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj589 testdata.508089 2026-02-17T12:20:41.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj590 testdata.508089 2026-02-17T12:20:41.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj591 testdata.508089 2026-02-17T12:20:41.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj592 testdata.508089 2026-02-17T12:20:41.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj593 testdata.508089 2026-02-17T12:20:41.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj594 testdata.508089 2026-02-17T12:20:41.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj595 testdata.508089 2026-02-17T12:20:41.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj596 testdata.508089 2026-02-17T12:20:41.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj597 testdata.508089 2026-02-17T12:20:41.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj598 testdata.508089 2026-02-17T12:20:41.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj599 testdata.508089 2026-02-17T12:20:41.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj600 testdata.508089 2026-02-17T12:20:41.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj601 testdata.508089 2026-02-17T12:20:41.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj602 testdata.508089 2026-02-17T12:20:41.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj603 testdata.508089 2026-02-17T12:20:41.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj604 testdata.508089 2026-02-17T12:20:41.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj605 testdata.508089 2026-02-17T12:20:41.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj606 testdata.508089 2026-02-17T12:20:41.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj607 testdata.508089 2026-02-17T12:20:41.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj608 testdata.508089 2026-02-17T12:20:41.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj609 testdata.508089 2026-02-17T12:20:41.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj610 testdata.508089 2026-02-17T12:20:41.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj611 testdata.508089 2026-02-17T12:20:41.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj612 testdata.508089 2026-02-17T12:20:41.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj613 testdata.508089 2026-02-17T12:20:41.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj614 testdata.508089 2026-02-17T12:20:41.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj615 testdata.508089 2026-02-17T12:20:41.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj616 testdata.508089 2026-02-17T12:20:41.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj617 testdata.508089 2026-02-17T12:20:41.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj618 testdata.508089 2026-02-17T12:20:41.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj619 testdata.508089 2026-02-17T12:20:41.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj620 testdata.508089 2026-02-17T12:20:41.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj621 testdata.508089 2026-02-17T12:20:41.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj622 testdata.508089 2026-02-17T12:20:41.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj623 testdata.508089 2026-02-17T12:20:41.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj624 testdata.508089 2026-02-17T12:20:41.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj625 testdata.508089 2026-02-17T12:20:41.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:41.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj626 testdata.508089 2026-02-17T12:20:42.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj627 testdata.508089 2026-02-17T12:20:42.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj628 testdata.508089 2026-02-17T12:20:42.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj629 testdata.508089 2026-02-17T12:20:42.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj630 testdata.508089 2026-02-17T12:20:42.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj631 testdata.508089 2026-02-17T12:20:42.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj632 testdata.508089 2026-02-17T12:20:42.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj633 testdata.508089 2026-02-17T12:20:42.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj634 testdata.508089 2026-02-17T12:20:42.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj635 testdata.508089 2026-02-17T12:20:42.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj636 testdata.508089 2026-02-17T12:20:42.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj637 testdata.508089 2026-02-17T12:20:42.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj638 testdata.508089 2026-02-17T12:20:42.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj639 testdata.508089 2026-02-17T12:20:42.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj640 testdata.508089 2026-02-17T12:20:42.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj641 testdata.508089 2026-02-17T12:20:42.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj642 testdata.508089 2026-02-17T12:20:42.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj643 testdata.508089 2026-02-17T12:20:42.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj644 testdata.508089 2026-02-17T12:20:42.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj645 testdata.508089 2026-02-17T12:20:42.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj646 testdata.508089 2026-02-17T12:20:42.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj647 testdata.508089 2026-02-17T12:20:42.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj648 testdata.508089 2026-02-17T12:20:42.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj649 testdata.508089 2026-02-17T12:20:42.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj650 testdata.508089 2026-02-17T12:20:42.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj651 testdata.508089 2026-02-17T12:20:42.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj652 testdata.508089 2026-02-17T12:20:42.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj653 testdata.508089 2026-02-17T12:20:42.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj654 testdata.508089 2026-02-17T12:20:42.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj655 testdata.508089 2026-02-17T12:20:42.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj656 testdata.508089 2026-02-17T12:20:42.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj657 testdata.508089 2026-02-17T12:20:42.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj658 testdata.508089 2026-02-17T12:20:42.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj659 testdata.508089 2026-02-17T12:20:42.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj660 testdata.508089 2026-02-17T12:20:42.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj661 testdata.508089 2026-02-17T12:20:42.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj662 testdata.508089 2026-02-17T12:20:42.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj663 testdata.508089 2026-02-17T12:20:42.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj664 testdata.508089 2026-02-17T12:20:42.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj665 testdata.508089 2026-02-17T12:20:42.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj666 testdata.508089 2026-02-17T12:20:42.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj667 testdata.508089 2026-02-17T12:20:42.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj668 testdata.508089 2026-02-17T12:20:42.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj669 testdata.508089 2026-02-17T12:20:42.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj670 testdata.508089 2026-02-17T12:20:42.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:42.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj671 testdata.508089 2026-02-17T12:20:43.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj672 testdata.508089 2026-02-17T12:20:43.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj673 testdata.508089 2026-02-17T12:20:43.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj674 testdata.508089 2026-02-17T12:20:43.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj675 testdata.508089 2026-02-17T12:20:43.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj676 testdata.508089 2026-02-17T12:20:43.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj677 testdata.508089 2026-02-17T12:20:43.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj678 testdata.508089 2026-02-17T12:20:43.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj679 testdata.508089 2026-02-17T12:20:43.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj680 testdata.508089 2026-02-17T12:20:43.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj681 testdata.508089 2026-02-17T12:20:43.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj682 testdata.508089 2026-02-17T12:20:43.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj683 testdata.508089 2026-02-17T12:20:43.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj684 testdata.508089 2026-02-17T12:20:43.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj685 testdata.508089 2026-02-17T12:20:43.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj686 testdata.508089 2026-02-17T12:20:43.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj687 testdata.508089 2026-02-17T12:20:43.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj688 testdata.508089 2026-02-17T12:20:43.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj689 testdata.508089 2026-02-17T12:20:43.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj690 testdata.508089 2026-02-17T12:20:43.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj691 testdata.508089 2026-02-17T12:20:43.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj692 testdata.508089 2026-02-17T12:20:43.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj693 testdata.508089 2026-02-17T12:20:43.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj694 testdata.508089 2026-02-17T12:20:43.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj695 testdata.508089 2026-02-17T12:20:43.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj696 testdata.508089 2026-02-17T12:20:43.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj697 testdata.508089 2026-02-17T12:20:43.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj698 testdata.508089 2026-02-17T12:20:43.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj699 testdata.508089 2026-02-17T12:20:43.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj700 testdata.508089 2026-02-17T12:20:43.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj701 testdata.508089 2026-02-17T12:20:43.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj702 testdata.508089 2026-02-17T12:20:43.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj703 testdata.508089 2026-02-17T12:20:43.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj704 testdata.508089 2026-02-17T12:20:43.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj705 testdata.508089 2026-02-17T12:20:43.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj706 testdata.508089 2026-02-17T12:20:43.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj707 testdata.508089 2026-02-17T12:20:43.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj708 testdata.508089 2026-02-17T12:20:43.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj709 testdata.508089 2026-02-17T12:20:43.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj710 testdata.508089 2026-02-17T12:20:43.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj711 testdata.508089 2026-02-17T12:20:43.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj712 testdata.508089 2026-02-17T12:20:43.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj713 testdata.508089 2026-02-17T12:20:43.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj714 testdata.508089 2026-02-17T12:20:43.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:43.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj715 testdata.508089 2026-02-17T12:20:44.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj716 testdata.508089 2026-02-17T12:20:44.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj717 testdata.508089 2026-02-17T12:20:44.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj718 testdata.508089 2026-02-17T12:20:44.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj719 testdata.508089 2026-02-17T12:20:44.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj720 testdata.508089 2026-02-17T12:20:44.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj721 testdata.508089 2026-02-17T12:20:44.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj722 testdata.508089 2026-02-17T12:20:44.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj723 testdata.508089 2026-02-17T12:20:44.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj724 testdata.508089 2026-02-17T12:20:44.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj725 testdata.508089 2026-02-17T12:20:44.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj726 testdata.508089 2026-02-17T12:20:44.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj727 testdata.508089 2026-02-17T12:20:44.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj728 testdata.508089 2026-02-17T12:20:44.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj729 testdata.508089 2026-02-17T12:20:44.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj730 testdata.508089 2026-02-17T12:20:44.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj731 testdata.508089 2026-02-17T12:20:44.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj732 testdata.508089 2026-02-17T12:20:44.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj733 testdata.508089 2026-02-17T12:20:44.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj734 testdata.508089 2026-02-17T12:20:44.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj735 testdata.508089 2026-02-17T12:20:44.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj736 testdata.508089 2026-02-17T12:20:44.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj737 testdata.508089 2026-02-17T12:20:44.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj738 testdata.508089 2026-02-17T12:20:44.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj739 testdata.508089 2026-02-17T12:20:44.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj740 testdata.508089 2026-02-17T12:20:44.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj741 testdata.508089 2026-02-17T12:20:44.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj742 testdata.508089 2026-02-17T12:20:44.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj743 testdata.508089 2026-02-17T12:20:44.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj744 testdata.508089 2026-02-17T12:20:44.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj745 testdata.508089 2026-02-17T12:20:44.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj746 testdata.508089 2026-02-17T12:20:44.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj747 testdata.508089 2026-02-17T12:20:44.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj748 testdata.508089 2026-02-17T12:20:44.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj749 testdata.508089 2026-02-17T12:20:44.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj750 testdata.508089 2026-02-17T12:20:44.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj751 testdata.508089 2026-02-17T12:20:44.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj752 testdata.508089 2026-02-17T12:20:44.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj753 testdata.508089 2026-02-17T12:20:44.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj754 testdata.508089 2026-02-17T12:20:44.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj755 testdata.508089 2026-02-17T12:20:44.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj756 testdata.508089 2026-02-17T12:20:44.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj757 testdata.508089 2026-02-17T12:20:44.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:44.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj758 testdata.508089 2026-02-17T12:20:45.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj759 testdata.508089 2026-02-17T12:20:45.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj760 testdata.508089 2026-02-17T12:20:45.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj761 testdata.508089 2026-02-17T12:20:45.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj762 testdata.508089 2026-02-17T12:20:45.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj763 testdata.508089 2026-02-17T12:20:45.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj764 testdata.508089 2026-02-17T12:20:45.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj765 testdata.508089 2026-02-17T12:20:45.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj766 testdata.508089 2026-02-17T12:20:45.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj767 testdata.508089 2026-02-17T12:20:45.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj768 testdata.508089 2026-02-17T12:20:45.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj769 testdata.508089 2026-02-17T12:20:45.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj770 testdata.508089 2026-02-17T12:20:45.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj771 testdata.508089 2026-02-17T12:20:45.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj772 testdata.508089 2026-02-17T12:20:45.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj773 testdata.508089 2026-02-17T12:20:45.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj774 testdata.508089 2026-02-17T12:20:45.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj775 testdata.508089 2026-02-17T12:20:45.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj776 testdata.508089 2026-02-17T12:20:45.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj777 testdata.508089 2026-02-17T12:20:45.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj778 testdata.508089 2026-02-17T12:20:45.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj779 testdata.508089 2026-02-17T12:20:45.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj780 testdata.508089 2026-02-17T12:20:45.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj781 testdata.508089 2026-02-17T12:20:45.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj782 testdata.508089 2026-02-17T12:20:45.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj783 testdata.508089 2026-02-17T12:20:45.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj784 testdata.508089 2026-02-17T12:20:45.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj785 testdata.508089 2026-02-17T12:20:45.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj786 testdata.508089 2026-02-17T12:20:45.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj787 testdata.508089 2026-02-17T12:20:45.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj788 testdata.508089 2026-02-17T12:20:45.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj789 testdata.508089 2026-02-17T12:20:45.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj790 testdata.508089 2026-02-17T12:20:45.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj791 testdata.508089 2026-02-17T12:20:45.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj792 testdata.508089 2026-02-17T12:20:45.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj793 testdata.508089 2026-02-17T12:20:45.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj794 testdata.508089 2026-02-17T12:20:45.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj795 testdata.508089 2026-02-17T12:20:45.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj796 testdata.508089 2026-02-17T12:20:45.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj797 testdata.508089 2026-02-17T12:20:45.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj798 testdata.508089 2026-02-17T12:20:45.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj799 testdata.508089 2026-02-17T12:20:45.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj800 testdata.508089 2026-02-17T12:20:45.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj801 testdata.508089 2026-02-17T12:20:45.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj802 testdata.508089 2026-02-17T12:20:45.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj803 testdata.508089 2026-02-17T12:20:45.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:45.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj804 testdata.508089 2026-02-17T12:20:46.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj805 testdata.508089 2026-02-17T12:20:46.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj806 testdata.508089 2026-02-17T12:20:46.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj807 testdata.508089 2026-02-17T12:20:46.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj808 testdata.508089 2026-02-17T12:20:46.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj809 testdata.508089 2026-02-17T12:20:46.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj810 testdata.508089 2026-02-17T12:20:46.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj811 testdata.508089 2026-02-17T12:20:46.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj812 testdata.508089 2026-02-17T12:20:46.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj813 testdata.508089 2026-02-17T12:20:46.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj814 testdata.508089 2026-02-17T12:20:46.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj815 testdata.508089 2026-02-17T12:20:46.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj816 testdata.508089 2026-02-17T12:20:46.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj817 testdata.508089 2026-02-17T12:20:46.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj818 testdata.508089 2026-02-17T12:20:46.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj819 testdata.508089 2026-02-17T12:20:46.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj820 testdata.508089 2026-02-17T12:20:46.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj821 testdata.508089 2026-02-17T12:20:46.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj822 testdata.508089 2026-02-17T12:20:46.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj823 testdata.508089 2026-02-17T12:20:46.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj824 testdata.508089 2026-02-17T12:20:46.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj825 testdata.508089 2026-02-17T12:20:46.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj826 testdata.508089 2026-02-17T12:20:46.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj827 testdata.508089 2026-02-17T12:20:46.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj828 testdata.508089 2026-02-17T12:20:46.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj829 testdata.508089 2026-02-17T12:20:46.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj830 testdata.508089 2026-02-17T12:20:46.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj831 testdata.508089 2026-02-17T12:20:46.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj832 testdata.508089 2026-02-17T12:20:46.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj833 testdata.508089 2026-02-17T12:20:46.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj834 testdata.508089 2026-02-17T12:20:46.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj835 testdata.508089 2026-02-17T12:20:46.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj836 testdata.508089 2026-02-17T12:20:46.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj837 testdata.508089 2026-02-17T12:20:46.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj838 testdata.508089 2026-02-17T12:20:46.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj839 testdata.508089 2026-02-17T12:20:46.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj840 testdata.508089 2026-02-17T12:20:46.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj841 testdata.508089 2026-02-17T12:20:46.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj842 testdata.508089 2026-02-17T12:20:46.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj843 testdata.508089 2026-02-17T12:20:46.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj844 testdata.508089 2026-02-17T12:20:46.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:46.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj845 testdata.508089 2026-02-17T12:20:47.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj846 testdata.508089 2026-02-17T12:20:47.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj847 testdata.508089 2026-02-17T12:20:47.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj848 testdata.508089 2026-02-17T12:20:47.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj849 testdata.508089 2026-02-17T12:20:47.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj850 testdata.508089 2026-02-17T12:20:47.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj851 testdata.508089 2026-02-17T12:20:47.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj852 testdata.508089 2026-02-17T12:20:47.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj853 testdata.508089 2026-02-17T12:20:47.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj854 testdata.508089 2026-02-17T12:20:47.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj855 testdata.508089 2026-02-17T12:20:47.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj856 testdata.508089 2026-02-17T12:20:47.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj857 testdata.508089 2026-02-17T12:20:47.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj858 testdata.508089 2026-02-17T12:20:47.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj859 testdata.508089 2026-02-17T12:20:47.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj860 testdata.508089 2026-02-17T12:20:47.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj861 testdata.508089 2026-02-17T12:20:47.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj862 testdata.508089 2026-02-17T12:20:47.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj863 testdata.508089 2026-02-17T12:20:47.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj864 testdata.508089 2026-02-17T12:20:47.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj865 testdata.508089 2026-02-17T12:20:47.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj866 testdata.508089 2026-02-17T12:20:47.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj867 testdata.508089 2026-02-17T12:20:47.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj868 testdata.508089 2026-02-17T12:20:47.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj869 testdata.508089 2026-02-17T12:20:47.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj870 testdata.508089 2026-02-17T12:20:47.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj871 testdata.508089 2026-02-17T12:20:47.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj872 testdata.508089 2026-02-17T12:20:47.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj873 testdata.508089 2026-02-17T12:20:47.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj874 testdata.508089 2026-02-17T12:20:47.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj875 testdata.508089 2026-02-17T12:20:47.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj876 testdata.508089 2026-02-17T12:20:47.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj877 testdata.508089 2026-02-17T12:20:47.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj878 testdata.508089 2026-02-17T12:20:47.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj879 testdata.508089 2026-02-17T12:20:47.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj880 testdata.508089 2026-02-17T12:20:47.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj881 testdata.508089 2026-02-17T12:20:47.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj882 testdata.508089 2026-02-17T12:20:47.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj883 testdata.508089 2026-02-17T12:20:47.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj884 testdata.508089 2026-02-17T12:20:47.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj885 testdata.508089 2026-02-17T12:20:47.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj886 testdata.508089 2026-02-17T12:20:47.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj887 testdata.508089 2026-02-17T12:20:47.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj888 testdata.508089 2026-02-17T12:20:47.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj889 testdata.508089 2026-02-17T12:20:47.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:47.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj890 testdata.508089 2026-02-17T12:20:48.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj891 testdata.508089 2026-02-17T12:20:48.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj892 testdata.508089 2026-02-17T12:20:48.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj893 testdata.508089 2026-02-17T12:20:48.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj894 testdata.508089 2026-02-17T12:20:48.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj895 testdata.508089 2026-02-17T12:20:48.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj896 testdata.508089 2026-02-17T12:20:48.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj897 testdata.508089 2026-02-17T12:20:48.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj898 testdata.508089 2026-02-17T12:20:48.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj899 testdata.508089 2026-02-17T12:20:48.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj900 testdata.508089 2026-02-17T12:20:48.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj901 testdata.508089 2026-02-17T12:20:48.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj902 testdata.508089 2026-02-17T12:20:48.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj903 testdata.508089 2026-02-17T12:20:48.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj904 testdata.508089 2026-02-17T12:20:48.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj905 testdata.508089 2026-02-17T12:20:48.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj906 testdata.508089 2026-02-17T12:20:48.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj907 testdata.508089 2026-02-17T12:20:48.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj908 testdata.508089 2026-02-17T12:20:48.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj909 testdata.508089 2026-02-17T12:20:48.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj910 testdata.508089 2026-02-17T12:20:48.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj911 testdata.508089 2026-02-17T12:20:48.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj912 testdata.508089 2026-02-17T12:20:48.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj913 testdata.508089 2026-02-17T12:20:48.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj914 testdata.508089 2026-02-17T12:20:48.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj915 testdata.508089 2026-02-17T12:20:48.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj916 testdata.508089 2026-02-17T12:20:48.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj917 testdata.508089 2026-02-17T12:20:48.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj918 testdata.508089 2026-02-17T12:20:48.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj919 testdata.508089 2026-02-17T12:20:48.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj920 testdata.508089 2026-02-17T12:20:48.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj921 testdata.508089 2026-02-17T12:20:48.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj922 testdata.508089 2026-02-17T12:20:48.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj923 testdata.508089 2026-02-17T12:20:48.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj924 testdata.508089 2026-02-17T12:20:48.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj925 testdata.508089 2026-02-17T12:20:48.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj926 testdata.508089 2026-02-17T12:20:48.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj927 testdata.508089 2026-02-17T12:20:48.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj928 testdata.508089 2026-02-17T12:20:48.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj929 testdata.508089 2026-02-17T12:20:48.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj930 testdata.508089 2026-02-17T12:20:48.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj931 testdata.508089 2026-02-17T12:20:48.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj932 testdata.508089 2026-02-17T12:20:48.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj933 testdata.508089 2026-02-17T12:20:48.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj934 testdata.508089 2026-02-17T12:20:48.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:48.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj935 testdata.508089 2026-02-17T12:20:49.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj936 testdata.508089 2026-02-17T12:20:49.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj937 testdata.508089 2026-02-17T12:20:49.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj938 testdata.508089 2026-02-17T12:20:49.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj939 testdata.508089 2026-02-17T12:20:49.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj940 testdata.508089 2026-02-17T12:20:49.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj941 testdata.508089 2026-02-17T12:20:49.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj942 testdata.508089 2026-02-17T12:20:49.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj943 testdata.508089 2026-02-17T12:20:49.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj944 testdata.508089 2026-02-17T12:20:49.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj945 testdata.508089 2026-02-17T12:20:49.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj946 testdata.508089 2026-02-17T12:20:49.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj947 testdata.508089 2026-02-17T12:20:49.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj948 testdata.508089 2026-02-17T12:20:49.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj949 testdata.508089 2026-02-17T12:20:49.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj950 testdata.508089 2026-02-17T12:20:49.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj951 testdata.508089 2026-02-17T12:20:49.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj952 testdata.508089 2026-02-17T12:20:49.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj953 testdata.508089 2026-02-17T12:20:49.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj954 testdata.508089 2026-02-17T12:20:49.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj955 testdata.508089 2026-02-17T12:20:49.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj956 testdata.508089 2026-02-17T12:20:49.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj957 testdata.508089 2026-02-17T12:20:49.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj958 testdata.508089 2026-02-17T12:20:49.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj959 testdata.508089 2026-02-17T12:20:49.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj960 testdata.508089 2026-02-17T12:20:49.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj961 testdata.508089 2026-02-17T12:20:49.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj962 testdata.508089 2026-02-17T12:20:49.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj963 testdata.508089 2026-02-17T12:20:49.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj964 testdata.508089 2026-02-17T12:20:49.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj965 testdata.508089 2026-02-17T12:20:49.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj966 testdata.508089 2026-02-17T12:20:49.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj967 testdata.508089 2026-02-17T12:20:49.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj968 testdata.508089 2026-02-17T12:20:49.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj969 testdata.508089 2026-02-17T12:20:49.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj970 testdata.508089 2026-02-17T12:20:49.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj971 testdata.508089 2026-02-17T12:20:49.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj972 testdata.508089 2026-02-17T12:20:49.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj973 testdata.508089 2026-02-17T12:20:49.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj974 testdata.508089 2026-02-17T12:20:49.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj975 testdata.508089 2026-02-17T12:20:49.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj976 testdata.508089 2026-02-17T12:20:49.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj977 testdata.508089 2026-02-17T12:20:49.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj978 testdata.508089 2026-02-17T12:20:49.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj979 testdata.508089 2026-02-17T12:20:49.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:49.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj980 testdata.508089 2026-02-17T12:20:50.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj981 testdata.508089 2026-02-17T12:20:50.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj982 testdata.508089 2026-02-17T12:20:50.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj983 testdata.508089 2026-02-17T12:20:50.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj984 testdata.508089 2026-02-17T12:20:50.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj985 testdata.508089 2026-02-17T12:20:50.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj986 testdata.508089 2026-02-17T12:20:50.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj987 testdata.508089 2026-02-17T12:20:50.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj988 testdata.508089 2026-02-17T12:20:50.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj989 testdata.508089 2026-02-17T12:20:50.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj990 testdata.508089 2026-02-17T12:20:50.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj991 testdata.508089 2026-02-17T12:20:50.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj992 testdata.508089 2026-02-17T12:20:50.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj993 testdata.508089 2026-02-17T12:20:50.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj994 testdata.508089 2026-02-17T12:20:50.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj995 testdata.508089 2026-02-17T12:20:50.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj996 testdata.508089 2026-02-17T12:20:50.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj997 testdata.508089 2026-02-17T12:20:50.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj998 testdata.508089 2026-02-17T12:20:50.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj999 testdata.508089 2026-02-17T12:20:50.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:20:50.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj1000 testdata.508089 2026-02-17T12:20:50.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:339: _scrub_abort: rm -f testdata.508089 2026-02-17T12:20:50.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:341: _scrub_abort: get_primary test obj1 2026-02-17T12:20:50.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T12:20:50.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T12:20:50.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T12:20:50.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:20:50.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:341: _scrub_abort: local primary=1 2026-02-17T12:20:50.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:342: _scrub_abort: local pgid=1.0 2026-02-17T12:20:50.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:344: _scrub_abort: ceph tell 1.0 schedule-deep-scrub 2026-02-17T12:20:50.584 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:20:50.584 INFO:tasks.workunit.client.0.trial128.stdout: "deep": true, 2026-02-17T12:20:50.584 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:20:50.585 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-03T12:19:10.578762+0000" 2026-02-17T12:20:50.585 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:20:50.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: set -o pipefail 2026-02-17T12:20:50.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:348: _scrub_abort: found=no 2026-02-17T12:20:50.592 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:349: _scrub_abort: seq 0 200 2026-02-17T12:20:50.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:349: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:20:50.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:351: _scrub_abort: flush_pg_stats 2026-02-17T12:20:50.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:20:50.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:20:50.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:20:50.745 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:20:50.745 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:20:50.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:20:50.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:50.746 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:20:50.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836525 2026-02-17T12:20:50.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836525 2026-02-17T12:20:50.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525' 2026-02-17T12:20:50.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:50.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:20:50.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705702 2026-02-17T12:20:50.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705702 2026-02-17T12:20:50.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705702' 2026-02-17T12:20:50.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:50.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:20:50.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574878 2026-02-17T12:20:50.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574878 2026-02-17T12:20:50.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836525 1-38654705702 2-55834574878' 2026-02-17T12:20:50.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:50.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836525 2026-02-17T12:20:50.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:50.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:20:50.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836525 2026-02-17T12:20:50.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:50.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836525 2026-02-17T12:20:50.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836525' 2026-02-17T12:20:50.960 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836525 2026-02-17T12:20:50.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:20:51.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836523 -lt 21474836525 2026-02-17T12:20:51.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:20:52.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:20:52.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:20:52.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836526 -lt 21474836525 2026-02-17T12:20:52.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:52.269 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705702 2026-02-17T12:20:52.269 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:52.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:20:52.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705702 2026-02-17T12:20:52.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:52.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705702 2026-02-17T12:20:52.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705702' 2026-02-17T12:20:52.273 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705702 2026-02-17T12:20:52.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:20:52.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705702 -lt 38654705702 2026-02-17T12:20:52.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:52.429 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574878 2026-02-17T12:20:52.429 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:52.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:20:52.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574878 2026-02-17T12:20:52.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:52.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574878 2026-02-17T12:20:52.432 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574878 2026-02-17T12:20:52.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574878' 2026-02-17T12:20:52.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:20:52.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574878 -lt 55834574878 2026-02-17T12:20:52.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: ceph pg dump pgs 2026-02-17T12:20:52.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: grep '^1.0' 2026-02-17T12:20:52.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: grep -q scrubbing 2026-02-17T12:20:52.730 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:20:52.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:349: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:20:52.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:351: _scrub_abort: flush_pg_stats 2026-02-17T12:20:52.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:20:52.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:20:52.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:20:52.887 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:20:52.887 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:20:52.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:20:52.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:52.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:20:52.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836528 2026-02-17T12:20:52.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836528 2026-02-17T12:20:52.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528' 2026-02-17T12:20:52.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:52.961 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:20:53.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705705 2026-02-17T12:20:53.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705705 2026-02-17T12:20:53.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705705' 2026-02-17T12:20:53.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:53.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:20:53.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574881 2026-02-17T12:20:53.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574881 2026-02-17T12:20:53.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836528 1-38654705705 2-55834574881' 2026-02-17T12:20:53.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:53.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836528 2026-02-17T12:20:53.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:53.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:20:53.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836528 2026-02-17T12:20:53.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:53.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836528 2026-02-17T12:20:53.107 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836528 2026-02-17T12:20:53.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836528' 2026-02-17T12:20:53.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:20:53.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836526 -lt 21474836528 2026-02-17T12:20:53.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:20:54.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:20:54.266 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:20:54.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836529 -lt 21474836528 2026-02-17T12:20:54.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:54.423 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705705 2026-02-17T12:20:54.423 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:54.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:20:54.425 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705705 2026-02-17T12:20:54.425 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:54.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705705 2026-02-17T12:20:54.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705705' 2026-02-17T12:20:54.427 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705705 2026-02-17T12:20:54.427 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:20:54.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705705 -lt 38654705705 2026-02-17T12:20:54.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:54.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574881 2026-02-17T12:20:54.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:54.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:20:54.582 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574881 2026-02-17T12:20:54.582 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:54.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574881 2026-02-17T12:20:54.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574881' 2026-02-17T12:20:54.584 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574881 2026-02-17T12:20:54.585 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:20:54.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574881 -lt 55834574881 2026-02-17T12:20:54.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: ceph pg dump pgs 2026-02-17T12:20:54.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: grep '^1.0' 2026-02-17T12:20:54.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: grep -q scrubbing 2026-02-17T12:20:54.876 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:20:54.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:349: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:20:54.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:351: _scrub_abort: flush_pg_stats 2026-02-17T12:20:54.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:20:54.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:20:55.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:20:55.032 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:20:55.032 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:20:55.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:20:55.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:55.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:20:55.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836531 2026-02-17T12:20:55.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836531 2026-02-17T12:20:55.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531' 2026-02-17T12:20:55.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:55.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:20:55.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705708 2026-02-17T12:20:55.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705708 2026-02-17T12:20:55.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531 1-38654705708' 2026-02-17T12:20:55.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:55.177 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:20:55.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574884 2026-02-17T12:20:55.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574884 2026-02-17T12:20:55.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836531 1-38654705708 2-55834574884' 2026-02-17T12:20:55.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:55.250 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836531 2026-02-17T12:20:55.250 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:55.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:20:55.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836531 2026-02-17T12:20:55.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:55.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836531 2026-02-17T12:20:55.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836531' 2026-02-17T12:20:55.254 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836531 2026-02-17T12:20:55.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:20:55.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836529 -lt 21474836531 2026-02-17T12:20:55.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:20:56.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:20:56.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:20:56.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836532 -lt 21474836531 2026-02-17T12:20:56.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:56.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705708 2026-02-17T12:20:56.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:56.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:20:56.570 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705708 2026-02-17T12:20:56.570 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:56.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705708 2026-02-17T12:20:56.572 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705708 2026-02-17T12:20:56.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705708' 2026-02-17T12:20:56.572 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:20:56.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705708 -lt 38654705708 2026-02-17T12:20:56.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:56.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574884 2026-02-17T12:20:56.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:56.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:20:56.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574884 2026-02-17T12:20:56.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:56.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574884 2026-02-17T12:20:56.729 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574884 2026-02-17T12:20:56.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574884' 2026-02-17T12:20:56.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:20:56.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574884 -lt 55834574884 2026-02-17T12:20:56.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: ceph pg dump pgs 2026-02-17T12:20:56.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: grep '^1.0' 2026-02-17T12:20:56.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: grep -q scrubbing 2026-02-17T12:20:57.033 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:20:57.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:354: _scrub_abort: found=yes 2026-02-17T12:20:57.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:356: _scrub_abort: break 2026-02-17T12:20:57.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:359: _scrub_abort: set +o pipefail 2026-02-17T12:20:57.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:361: _scrub_abort: test yes = no 2026-02-17T12:20:57.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:367: _scrub_abort: ceph osd set nodeep-scrub 2026-02-17T12:20:57.234 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:20:57.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:368: _scrub_abort: '[' deep-scrub = deep-scrub ']' 2026-02-17T12:20:57.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:370: _scrub_abort: ceph osd set noscrub 2026-02-17T12:20:57.438 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:20:57.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:374: _scrub_abort: set -o pipefail 2026-02-17T12:20:57.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: seq 0 200 2026-02-17T12:20:57.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:20:57.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:20:57.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:20:57.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:20:57.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:20:57.606 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:20:57.606 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:20:57.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:20:57.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:57.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:20:57.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836535 2026-02-17T12:20:57.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836535 2026-02-17T12:20:57.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535' 2026-02-17T12:20:57.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:57.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:20:57.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705712 2026-02-17T12:20:57.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705712 2026-02-17T12:20:57.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705712' 2026-02-17T12:20:57.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:20:57.750 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:20:57.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574888 2026-02-17T12:20:57.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574888 2026-02-17T12:20:57.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836535 1-38654705712 2-55834574888' 2026-02-17T12:20:57.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:20:57.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836535 2026-02-17T12:20:57.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:20:57.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:20:57.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836535 2026-02-17T12:20:57.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:20:57.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836535 2026-02-17T12:20:57.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836535' 2026-02-17T12:20:57.826 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836535 2026-02-17T12:20:57.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:20:57.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836534 -lt 21474836535 2026-02-17T12:20:57.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:20:58.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:20:58.986 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:20:59.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836534 -lt 21474836535 2026-02-17T12:20:59.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:00.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:21:00.144 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:00.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836537 -lt 21474836535 2026-02-17T12:21:00.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:00.294 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705712 2026-02-17T12:21:00.295 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:00.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:00.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705712 2026-02-17T12:21:00.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:00.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705712 2026-02-17T12:21:00.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705712' 2026-02-17T12:21:00.298 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705712 2026-02-17T12:21:00.299 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:00.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705713 -lt 38654705712 2026-02-17T12:21:00.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:00.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574888 2026-02-17T12:21:00.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:00.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:00.464 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574888 2026-02-17T12:21:00.464 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:00.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574888 2026-02-17T12:21:00.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574888' 2026-02-17T12:21:00.466 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574888 2026-02-17T12:21:00.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:00.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574889 -lt 55834574888 2026-02-17T12:21:00.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:00.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:00.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:00.761 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:00.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:00.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:00.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:00.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:00.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:00.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:00.917 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:00.917 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:00.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:00.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:00.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:00.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836539 2026-02-17T12:21:00.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836539 2026-02-17T12:21:00.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836539' 2026-02-17T12:21:00.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:00.990 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:01.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705716 2026-02-17T12:21:01.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705716 2026-02-17T12:21:01.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836539 1-38654705716' 2026-02-17T12:21:01.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:01.060 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:01.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574892 2026-02-17T12:21:01.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574892 2026-02-17T12:21:01.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836539 1-38654705716 2-55834574892' 2026-02-17T12:21:01.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:01.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836539 2026-02-17T12:21:01.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:01.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:01.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836539 2026-02-17T12:21:01.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:01.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836539 2026-02-17T12:21:01.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836539' 2026-02-17T12:21:01.137 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836539 2026-02-17T12:21:01.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:01.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836537 -lt 21474836539 2026-02-17T12:21:01.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:02.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:02.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:02.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836540 -lt 21474836539 2026-02-17T12:21:02.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:02.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705716 2026-02-17T12:21:02.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:02.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:02.455 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705716 2026-02-17T12:21:02.455 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:02.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705716 2026-02-17T12:21:02.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705716' 2026-02-17T12:21:02.457 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705716 2026-02-17T12:21:02.457 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:02.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705716 -lt 38654705716 2026-02-17T12:21:02.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:02.614 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574892 2026-02-17T12:21:02.614 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:02.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:02.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574892 2026-02-17T12:21:02.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:02.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574892 2026-02-17T12:21:02.617 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574892 2026-02-17T12:21:02.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574892' 2026-02-17T12:21:02.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:02.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574892 -lt 55834574892 2026-02-17T12:21:02.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:02.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:02.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:02.920 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:02.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:02.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:02.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:02.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:02.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:03.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:03.084 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:03.085 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:03.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:03.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:03.085 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:03.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836543 2026-02-17T12:21:03.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836543 2026-02-17T12:21:03.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836543' 2026-02-17T12:21:03.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:03.156 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:03.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705719 2026-02-17T12:21:03.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705719 2026-02-17T12:21:03.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836543 1-38654705719' 2026-02-17T12:21:03.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:03.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:03.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574895 2026-02-17T12:21:03.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574895 2026-02-17T12:21:03.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836543 1-38654705719 2-55834574895' 2026-02-17T12:21:03.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:03.302 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836543 2026-02-17T12:21:03.302 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:03.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:03.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836543 2026-02-17T12:21:03.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:03.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836543 2026-02-17T12:21:03.306 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836543 2026-02-17T12:21:03.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836543' 2026-02-17T12:21:03.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:03.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836540 -lt 21474836543 2026-02-17T12:21:03.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:04.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:04.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:04.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836543 -lt 21474836543 2026-02-17T12:21:04.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:04.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705719 2026-02-17T12:21:04.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:04.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:04.625 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705719 2026-02-17T12:21:04.625 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:04.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705719 2026-02-17T12:21:04.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705719' 2026-02-17T12:21:04.626 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705719 2026-02-17T12:21:04.627 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:04.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705719 -lt 38654705719 2026-02-17T12:21:04.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:04.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574895 2026-02-17T12:21:04.793 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:04.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:04.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574895 2026-02-17T12:21:04.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:04.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574895 2026-02-17T12:21:04.796 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574895 2026-02-17T12:21:04.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574895' 2026-02-17T12:21:04.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:04.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574895 -lt 55834574895 2026-02-17T12:21:04.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:04.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:04.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:05.102 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:05.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:05.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:05.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:05.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:05.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:05.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:05.264 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:05.264 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:05.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:05.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:05.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:05.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836546 2026-02-17T12:21:05.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836546 2026-02-17T12:21:05.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836546' 2026-02-17T12:21:05.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:05.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:05.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705722 2026-02-17T12:21:05.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705722 2026-02-17T12:21:05.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836546 1-38654705722' 2026-02-17T12:21:05.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:05.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:05.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574898 2026-02-17T12:21:05.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574898 2026-02-17T12:21:05.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836546 1-38654705722 2-55834574898' 2026-02-17T12:21:05.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:05.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836546 2026-02-17T12:21:05.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:05.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:05.480 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836546 2026-02-17T12:21:05.480 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:05.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836546 2026-02-17T12:21:05.481 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836546 2026-02-17T12:21:05.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836546' 2026-02-17T12:21:05.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:05.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836546 -lt 21474836546 2026-02-17T12:21:05.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:05.646 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705722 2026-02-17T12:21:05.646 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:05.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:05.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705722 2026-02-17T12:21:05.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:05.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705722 2026-02-17T12:21:05.649 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705722 2026-02-17T12:21:05.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705722' 2026-02-17T12:21:05.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:05.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705722 -lt 38654705722 2026-02-17T12:21:05.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:05.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574898 2026-02-17T12:21:05.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:05.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:05.812 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574898 2026-02-17T12:21:05.813 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:05.814 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574898 2026-02-17T12:21:05.814 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574898 2026-02-17T12:21:05.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574898' 2026-02-17T12:21:05.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:05.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574898 -lt 55834574898 2026-02-17T12:21:05.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:05.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:05.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:06.115 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:06.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:06.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:06.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:06.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:06.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:06.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:06.270 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:06.270 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:06.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:06.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:06.270 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:06.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836548 2026-02-17T12:21:06.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836548 2026-02-17T12:21:06.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836548' 2026-02-17T12:21:06.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:06.347 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:06.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705724 2026-02-17T12:21:06.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705724 2026-02-17T12:21:06.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836548 1-38654705724' 2026-02-17T12:21:06.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:06.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:06.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574900 2026-02-17T12:21:06.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574900 2026-02-17T12:21:06.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836548 1-38654705724 2-55834574900' 2026-02-17T12:21:06.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:06.488 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836548 2026-02-17T12:21:06.488 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:06.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:06.490 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836548 2026-02-17T12:21:06.490 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:06.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836548 2026-02-17T12:21:06.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836548' 2026-02-17T12:21:06.492 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836548 2026-02-17T12:21:06.492 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:06.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836546 -lt 21474836548 2026-02-17T12:21:06.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:07.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:07.649 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:07.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836549 -lt 21474836548 2026-02-17T12:21:07.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:07.807 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705724 2026-02-17T12:21:07.807 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:07.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:07.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705724 2026-02-17T12:21:07.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:07.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705724 2026-02-17T12:21:07.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705724' 2026-02-17T12:21:07.811 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705724 2026-02-17T12:21:07.811 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:07.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705725 -lt 38654705724 2026-02-17T12:21:07.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:07.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574900 2026-02-17T12:21:07.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:07.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:07.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574900 2026-02-17T12:21:07.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:07.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574900 2026-02-17T12:21:07.973 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574900 2026-02-17T12:21:07.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574900' 2026-02-17T12:21:07.974 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:08.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574901 -lt 55834574900 2026-02-17T12:21:08.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:08.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:08.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:08.267 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:08.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:08.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:08.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:08.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:08.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:08.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:08.429 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:08.429 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:08.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:08.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:08.430 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:08.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836551 2026-02-17T12:21:08.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836551 2026-02-17T12:21:08.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836551' 2026-02-17T12:21:08.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:08.502 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:08.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705727 2026-02-17T12:21:08.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705727 2026-02-17T12:21:08.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836551 1-38654705727' 2026-02-17T12:21:08.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:08.574 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:08.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574904 2026-02-17T12:21:08.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574904 2026-02-17T12:21:08.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836551 1-38654705727 2-55834574904' 2026-02-17T12:21:08.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:08.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836551 2026-02-17T12:21:08.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:08.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:08.646 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836551 2026-02-17T12:21:08.646 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:08.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836551 2026-02-17T12:21:08.648 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836551 2026-02-17T12:21:08.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836551' 2026-02-17T12:21:08.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:08.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836549 -lt 21474836551 2026-02-17T12:21:08.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:09.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:09.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:09.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836552 -lt 21474836551 2026-02-17T12:21:09.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:09.954 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705727 2026-02-17T12:21:09.954 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:09.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:09.956 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705727 2026-02-17T12:21:09.956 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:09.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705727 2026-02-17T12:21:09.958 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705727 2026-02-17T12:21:09.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705727' 2026-02-17T12:21:09.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:10.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705728 -lt 38654705727 2026-02-17T12:21:10.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:10.114 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574904 2026-02-17T12:21:10.114 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:10.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:10.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574904 2026-02-17T12:21:10.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:10.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574904 2026-02-17T12:21:10.118 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574904 2026-02-17T12:21:10.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574904' 2026-02-17T12:21:10.118 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:10.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574904 -lt 55834574904 2026-02-17T12:21:10.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:10.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:10.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:10.422 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:10.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:10.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:10.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:10.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:10.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:10.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:10.585 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:10.585 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:10.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:10.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:10.585 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:10.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836554 2026-02-17T12:21:10.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836554 2026-02-17T12:21:10.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836554' 2026-02-17T12:21:10.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:10.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:10.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705731 2026-02-17T12:21:10.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705731 2026-02-17T12:21:10.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836554 1-38654705731' 2026-02-17T12:21:10.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:10.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:10.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574907 2026-02-17T12:21:10.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574907 2026-02-17T12:21:10.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836554 1-38654705731 2-55834574907' 2026-02-17T12:21:10.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:10.801 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836554 2026-02-17T12:21:10.801 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:10.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:10.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836554 2026-02-17T12:21:10.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:10.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836554 2026-02-17T12:21:10.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836554' 2026-02-17T12:21:10.804 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836554 2026-02-17T12:21:10.805 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:10.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836552 -lt 21474836554 2026-02-17T12:21:10.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:11.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:11.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:12.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836555 -lt 21474836554 2026-02-17T12:21:12.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:12.111 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705731 2026-02-17T12:21:12.111 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:12.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:12.113 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705731 2026-02-17T12:21:12.113 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:12.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705731 2026-02-17T12:21:12.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705731' 2026-02-17T12:21:12.115 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705731 2026-02-17T12:21:12.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:12.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705731 -lt 38654705731 2026-02-17T12:21:12.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:12.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574907 2026-02-17T12:21:12.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:12.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:12.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574907 2026-02-17T12:21:12.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:12.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574907 2026-02-17T12:21:12.277 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574907 2026-02-17T12:21:12.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574907' 2026-02-17T12:21:12.278 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:12.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574907 -lt 55834574907 2026-02-17T12:21:12.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:12.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:12.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:12.582 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:12.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:12.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:12.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:12.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:12.592 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:12.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:12.742 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:12.743 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:12.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:12.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:12.743 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:12.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836557 2026-02-17T12:21:12.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836557 2026-02-17T12:21:12.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836557' 2026-02-17T12:21:12.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:12.812 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:12.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705734 2026-02-17T12:21:12.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705734 2026-02-17T12:21:12.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836557 1-38654705734' 2026-02-17T12:21:12.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:12.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:12.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574910 2026-02-17T12:21:12.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574910 2026-02-17T12:21:12.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836557 1-38654705734 2-55834574910' 2026-02-17T12:21:12.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:12.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836557 2026-02-17T12:21:12.954 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:12.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:12.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836557 2026-02-17T12:21:12.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:12.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836557 2026-02-17T12:21:12.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836557' 2026-02-17T12:21:12.957 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836557 2026-02-17T12:21:12.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:13.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836555 -lt 21474836557 2026-02-17T12:21:13.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:14.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:14.121 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:14.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836558 -lt 21474836557 2026-02-17T12:21:14.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:14.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705734 2026-02-17T12:21:14.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:14.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:14.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705734 2026-02-17T12:21:14.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:14.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705734 2026-02-17T12:21:14.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705734' 2026-02-17T12:21:14.277 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705734 2026-02-17T12:21:14.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:14.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705734 -lt 38654705734 2026-02-17T12:21:14.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:14.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574910 2026-02-17T12:21:14.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:14.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:14.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574910 2026-02-17T12:21:14.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:14.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574910 2026-02-17T12:21:14.440 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574910 2026-02-17T12:21:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574910' 2026-02-17T12:21:14.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:14.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574910 -lt 55834574910 2026-02-17T12:21:14.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:14.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:14.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:14.737 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:14.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:14.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:14.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:14.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:14.747 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:14.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:14.898 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:14.898 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:14.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:14.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:14.898 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:14.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836560 2026-02-17T12:21:14.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836560 2026-02-17T12:21:14.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836560' 2026-02-17T12:21:14.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:14.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:15.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705737 2026-02-17T12:21:15.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705737 2026-02-17T12:21:15.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836560 1-38654705737' 2026-02-17T12:21:15.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:15.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:15.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574913 2026-02-17T12:21:15.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574913 2026-02-17T12:21:15.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836560 1-38654705737 2-55834574913' 2026-02-17T12:21:15.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:15.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836560 2026-02-17T12:21:15.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:15.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:15.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836560 2026-02-17T12:21:15.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:15.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836560 2026-02-17T12:21:15.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836560' 2026-02-17T12:21:15.118 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836560 2026-02-17T12:21:15.119 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:15.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836558 -lt 21474836560 2026-02-17T12:21:15.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:16.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:16.284 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:16.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836561 -lt 21474836560 2026-02-17T12:21:16.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:16.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705737 2026-02-17T12:21:16.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:16.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:16.435 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705737 2026-02-17T12:21:16.435 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:16.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705737 2026-02-17T12:21:16.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705737' 2026-02-17T12:21:16.437 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705737 2026-02-17T12:21:16.437 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:16.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705737 -lt 38654705737 2026-02-17T12:21:16.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:16.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574913 2026-02-17T12:21:16.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:16.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:16.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574913 2026-02-17T12:21:16.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:16.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574913 2026-02-17T12:21:16.604 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574913 2026-02-17T12:21:16.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574913' 2026-02-17T12:21:16.605 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:16.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574913 -lt 55834574913 2026-02-17T12:21:16.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:16.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:16.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:16.906 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:16.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:16.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:16.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:16.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:16.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:17.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:17.068 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:17.069 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:17.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:17.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:17.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:17.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836564 2026-02-17T12:21:17.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836564 2026-02-17T12:21:17.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836564' 2026-02-17T12:21:17.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:17.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:17.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705740 2026-02-17T12:21:17.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705740 2026-02-17T12:21:17.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836564 1-38654705740' 2026-02-17T12:21:17.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:17.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:17.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574916 2026-02-17T12:21:17.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574916 2026-02-17T12:21:17.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836564 1-38654705740 2-55834574916' 2026-02-17T12:21:17.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:17.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836564 2026-02-17T12:21:17.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:17.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:17.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836564 2026-02-17T12:21:17.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:17.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836564 2026-02-17T12:21:17.284 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836564 2026-02-17T12:21:17.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836564' 2026-02-17T12:21:17.285 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:17.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836561 -lt 21474836564 2026-02-17T12:21:17.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:18.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:18.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:18.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836564 -lt 21474836564 2026-02-17T12:21:18.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:18.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705740 2026-02-17T12:21:18.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:18.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:18.596 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705740 2026-02-17T12:21:18.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:18.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705740 2026-02-17T12:21:18.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705740' 2026-02-17T12:21:18.598 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705740 2026-02-17T12:21:18.599 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:18.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705740 -lt 38654705740 2026-02-17T12:21:18.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:18.750 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574916 2026-02-17T12:21:18.750 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:18.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:18.752 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574916 2026-02-17T12:21:18.752 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:18.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574916 2026-02-17T12:21:18.753 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574916 2026-02-17T12:21:18.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574916' 2026-02-17T12:21:18.754 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:18.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574916 -lt 55834574916 2026-02-17T12:21:18.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:18.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:18.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:19.055 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:19.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:19.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:19.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:19.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:19.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:19.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:19.218 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:19.218 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:19.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:19.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:19.219 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:19.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836567 2026-02-17T12:21:19.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836567 2026-02-17T12:21:19.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836567' 2026-02-17T12:21:19.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:19.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:19.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705743 2026-02-17T12:21:19.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705743 2026-02-17T12:21:19.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836567 1-38654705743' 2026-02-17T12:21:19.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:19.361 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:19.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574919 2026-02-17T12:21:19.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574919 2026-02-17T12:21:19.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836567 1-38654705743 2-55834574919' 2026-02-17T12:21:19.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:19.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836567 2026-02-17T12:21:19.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:19.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:19.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836567 2026-02-17T12:21:19.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:19.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836567 2026-02-17T12:21:19.435 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836567 2026-02-17T12:21:19.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836567' 2026-02-17T12:21:19.435 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:19.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836567 -lt 21474836567 2026-02-17T12:21:19.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:19.592 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705743 2026-02-17T12:21:19.592 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:19.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:19.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705743 2026-02-17T12:21:19.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:19.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705743 2026-02-17T12:21:19.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705743' 2026-02-17T12:21:19.596 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705743 2026-02-17T12:21:19.596 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:19.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705743 -lt 38654705743 2026-02-17T12:21:19.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:19.754 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574919 2026-02-17T12:21:19.754 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:19.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:19.756 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574919 2026-02-17T12:21:19.756 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:19.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574919 2026-02-17T12:21:19.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574919' 2026-02-17T12:21:19.758 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574919 2026-02-17T12:21:19.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:19.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574919 -lt 55834574919 2026-02-17T12:21:19.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:19.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:19.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:20.055 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:20.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:20.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:20.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:20.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:20.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:20.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:20.222 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:20.222 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:20.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:20.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:20.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:20.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836569 2026-02-17T12:21:20.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836569 2026-02-17T12:21:20.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836569' 2026-02-17T12:21:20.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:20.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:20.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705745 2026-02-17T12:21:20.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705745 2026-02-17T12:21:20.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836569 1-38654705745' 2026-02-17T12:21:20.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:20.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:20.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574921 2026-02-17T12:21:20.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574921 2026-02-17T12:21:20.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836569 1-38654705745 2-55834574921' 2026-02-17T12:21:20.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:20.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836569 2026-02-17T12:21:20.437 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:20.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:20.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836569 2026-02-17T12:21:20.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:20.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836569 2026-02-17T12:21:20.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836569' 2026-02-17T12:21:20.440 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836569 2026-02-17T12:21:20.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:20.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836567 -lt 21474836569 2026-02-17T12:21:20.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:21.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:21.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:21.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836570 -lt 21474836569 2026-02-17T12:21:21.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:21.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705745 2026-02-17T12:21:21.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:21.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:21.750 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705745 2026-02-17T12:21:21.750 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:21.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705745 2026-02-17T12:21:21.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705745' 2026-02-17T12:21:21.752 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705745 2026-02-17T12:21:21.752 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:21.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705746 -lt 38654705745 2026-02-17T12:21:21.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:21.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574921 2026-02-17T12:21:21.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:21.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:21.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574921 2026-02-17T12:21:21.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:21.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574921 2026-02-17T12:21:21.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574921' 2026-02-17T12:21:21.919 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574921 2026-02-17T12:21:21.920 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:22.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574922 -lt 55834574921 2026-02-17T12:21:22.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:22.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:22.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:22.215 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:22.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:22.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:22.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:22.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:22.225 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:22.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:22.378 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:22.378 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:22.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:22.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:22.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:22.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836572 2026-02-17T12:21:22.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836572 2026-02-17T12:21:22.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836572' 2026-02-17T12:21:22.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:22.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:22.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705748 2026-02-17T12:21:22.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705748 2026-02-17T12:21:22.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836572 1-38654705748' 2026-02-17T12:21:22.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:22.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:22.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574924 2026-02-17T12:21:22.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574924 2026-02-17T12:21:22.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836572 1-38654705748 2-55834574924' 2026-02-17T12:21:22.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:22.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836572 2026-02-17T12:21:22.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:22.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:22.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836572 2026-02-17T12:21:22.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:22.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836572 2026-02-17T12:21:22.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836572' 2026-02-17T12:21:22.597 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836572 2026-02-17T12:21:22.597 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:22.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836570 -lt 21474836572 2026-02-17T12:21:22.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:23.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:23.747 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:23.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836573 -lt 21474836572 2026-02-17T12:21:23.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:23.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705748 2026-02-17T12:21:23.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:23.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:23.906 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705748 2026-02-17T12:21:23.906 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:23.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705748 2026-02-17T12:21:23.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705748' 2026-02-17T12:21:23.907 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705748 2026-02-17T12:21:23.908 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:24.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705749 -lt 38654705748 2026-02-17T12:21:24.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:24.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574924 2026-02-17T12:21:24.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:24.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:24.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574924 2026-02-17T12:21:24.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:24.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574924 2026-02-17T12:21:24.073 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574924 2026-02-17T12:21:24.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574924' 2026-02-17T12:21:24.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:24.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574925 -lt 55834574924 2026-02-17T12:21:24.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:24.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:24.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:24.377 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:24.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:24.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:24.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:24.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:24.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:24.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:24.537 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:24.537 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:24.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:24.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:24.538 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:24.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836575 2026-02-17T12:21:24.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836575 2026-02-17T12:21:24.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836575' 2026-02-17T12:21:24.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:24.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:24.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705751 2026-02-17T12:21:24.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705751 2026-02-17T12:21:24.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836575 1-38654705751' 2026-02-17T12:21:24.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:24.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:24.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574928 2026-02-17T12:21:24.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574928 2026-02-17T12:21:24.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836575 1-38654705751 2-55834574928' 2026-02-17T12:21:24.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:24.754 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836575 2026-02-17T12:21:24.754 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:24.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:24.755 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836575 2026-02-17T12:21:24.755 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:24.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836575 2026-02-17T12:21:24.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836575' 2026-02-17T12:21:24.757 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836575 2026-02-17T12:21:24.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:24.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836573 -lt 21474836575 2026-02-17T12:21:24.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:25.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:25.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:26.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836576 -lt 21474836575 2026-02-17T12:21:26.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:26.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705751 2026-02-17T12:21:26.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:26.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:26.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705751 2026-02-17T12:21:26.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:26.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705751 2026-02-17T12:21:26.076 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705751 2026-02-17T12:21:26.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705751' 2026-02-17T12:21:26.077 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:26.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705752 -lt 38654705751 2026-02-17T12:21:26.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:26.225 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574928 2026-02-17T12:21:26.225 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:26.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:26.227 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574928 2026-02-17T12:21:26.227 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:26.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574928 2026-02-17T12:21:26.228 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574928 2026-02-17T12:21:26.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574928' 2026-02-17T12:21:26.229 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:26.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574928 -lt 55834574928 2026-02-17T12:21:26.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:26.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:26.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:26.529 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:26.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:26.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:26.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:26.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:26.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:26.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:26.692 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:26.692 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:26.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:26.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:26.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:26.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836578 2026-02-17T12:21:26.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836578 2026-02-17T12:21:26.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836578' 2026-02-17T12:21:26.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:26.762 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:26.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705755 2026-02-17T12:21:26.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705755 2026-02-17T12:21:26.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836578 1-38654705755' 2026-02-17T12:21:26.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:26.834 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:26.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574931 2026-02-17T12:21:26.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574931 2026-02-17T12:21:26.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836578 1-38654705755 2-55834574931' 2026-02-17T12:21:26.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:26.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836578 2026-02-17T12:21:26.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:26.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:26.907 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836578 2026-02-17T12:21:26.907 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:26.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836578 2026-02-17T12:21:26.908 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836578 2026-02-17T12:21:26.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836578' 2026-02-17T12:21:26.909 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:27.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836576 -lt 21474836578 2026-02-17T12:21:27.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:28.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:28.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:28.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836579 -lt 21474836578 2026-02-17T12:21:28.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:28.215 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705755 2026-02-17T12:21:28.216 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:28.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:28.216 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705755 2026-02-17T12:21:28.217 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:28.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705755 2026-02-17T12:21:28.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705755' 2026-02-17T12:21:28.219 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705755 2026-02-17T12:21:28.219 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:28.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705755 -lt 38654705755 2026-02-17T12:21:28.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:28.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574931 2026-02-17T12:21:28.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:28.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:28.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574931 2026-02-17T12:21:28.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:28.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574931 2026-02-17T12:21:28.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574931' 2026-02-17T12:21:28.386 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574931 2026-02-17T12:21:28.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:28.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574931 -lt 55834574931 2026-02-17T12:21:28.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:28.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:28.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:28.691 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:28.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:28.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:28.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:28.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:28.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:28.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:28.855 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:28.855 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:28.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:28.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:28.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:28.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836581 2026-02-17T12:21:28.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836581 2026-02-17T12:21:28.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836581' 2026-02-17T12:21:28.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:28.925 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:28.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705758 2026-02-17T12:21:28.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705758 2026-02-17T12:21:28.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836581 1-38654705758' 2026-02-17T12:21:28.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:28.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:29.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574934 2026-02-17T12:21:29.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574934 2026-02-17T12:21:29.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836581 1-38654705758 2-55834574934' 2026-02-17T12:21:29.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:29.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836581 2026-02-17T12:21:29.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:29.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:29.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836581 2026-02-17T12:21:29.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:29.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836581 2026-02-17T12:21:29.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836581' 2026-02-17T12:21:29.073 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836581 2026-02-17T12:21:29.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:29.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836579 -lt 21474836581 2026-02-17T12:21:29.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:30.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:30.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:30.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836582 -lt 21474836581 2026-02-17T12:21:30.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:30.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705758 2026-02-17T12:21:30.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:30.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:30.380 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705758 2026-02-17T12:21:30.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:30.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705758 2026-02-17T12:21:30.382 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705758 2026-02-17T12:21:30.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705758' 2026-02-17T12:21:30.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:30.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705758 -lt 38654705758 2026-02-17T12:21:30.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:30.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574934 2026-02-17T12:21:30.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:30.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:30.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574934 2026-02-17T12:21:30.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:30.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574934 2026-02-17T12:21:30.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574934' 2026-02-17T12:21:30.550 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574934 2026-02-17T12:21:30.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:30.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574934 -lt 55834574934 2026-02-17T12:21:30.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:30.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:30.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:30.851 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:30.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:30.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:30.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:30.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:30.861 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:31.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:31.016 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:31.016 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:31.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:31.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:31.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:31.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836584 2026-02-17T12:21:31.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836584 2026-02-17T12:21:31.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836584' 2026-02-17T12:21:31.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:31.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:31.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705761 2026-02-17T12:21:31.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705761 2026-02-17T12:21:31.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836584 1-38654705761' 2026-02-17T12:21:31.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:31.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:31.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574937 2026-02-17T12:21:31.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574937 2026-02-17T12:21:31.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836584 1-38654705761 2-55834574937' 2026-02-17T12:21:31.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:31.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836584 2026-02-17T12:21:31.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:31.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:31.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836584 2026-02-17T12:21:31.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:31.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836584 2026-02-17T12:21:31.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836584' 2026-02-17T12:21:31.234 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836584 2026-02-17T12:21:31.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:31.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836582 -lt 21474836584 2026-02-17T12:21:31.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:32.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:32.396 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:32.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836585 -lt 21474836584 2026-02-17T12:21:32.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:32.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705761 2026-02-17T12:21:32.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:32.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:32.549 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705761 2026-02-17T12:21:32.549 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:32.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705761 2026-02-17T12:21:32.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705761' 2026-02-17T12:21:32.550 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705761 2026-02-17T12:21:32.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:32.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705761 -lt 38654705761 2026-02-17T12:21:32.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:32.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574937 2026-02-17T12:21:32.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:32.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:32.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574937 2026-02-17T12:21:32.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:32.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574937 2026-02-17T12:21:32.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574937' 2026-02-17T12:21:32.711 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574937 2026-02-17T12:21:32.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:32.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574937 -lt 55834574937 2026-02-17T12:21:32.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:32.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:32.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:33.013 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:33.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:33.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:33.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:33.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:33.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:33.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:33.176 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:33.176 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:33.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:33.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:33.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:33.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836588 2026-02-17T12:21:33.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836588 2026-02-17T12:21:33.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836588' 2026-02-17T12:21:33.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:33.243 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:33.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705764 2026-02-17T12:21:33.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705764 2026-02-17T12:21:33.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836588 1-38654705764' 2026-02-17T12:21:33.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:33.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:33.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574940 2026-02-17T12:21:33.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574940 2026-02-17T12:21:33.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836588 1-38654705764 2-55834574940' 2026-02-17T12:21:33.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:33.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836588 2026-02-17T12:21:33.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:33.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:33.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836588 2026-02-17T12:21:33.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:33.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836588 2026-02-17T12:21:33.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836588' 2026-02-17T12:21:33.390 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836588 2026-02-17T12:21:33.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:33.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836588 -lt 21474836588 2026-02-17T12:21:33.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:33.549 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705764 2026-02-17T12:21:33.549 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:33.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:33.551 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705764 2026-02-17T12:21:33.551 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:33.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705764 2026-02-17T12:21:33.552 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705764 2026-02-17T12:21:33.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705764' 2026-02-17T12:21:33.553 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:33.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705764 -lt 38654705764 2026-02-17T12:21:33.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:33.707 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574940 2026-02-17T12:21:33.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:33.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:33.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574940 2026-02-17T12:21:33.709 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:33.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574940 2026-02-17T12:21:33.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574940' 2026-02-17T12:21:33.711 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574940 2026-02-17T12:21:33.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:33.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574940 -lt 55834574940 2026-02-17T12:21:33.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:33.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:33.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:34.007 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:34.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:34.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:34.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:34.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:34.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:34.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:34.163 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:34.163 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:34.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:34.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:34.163 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:34.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836590 2026-02-17T12:21:34.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836590 2026-02-17T12:21:34.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836590' 2026-02-17T12:21:34.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:34.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:34.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705766 2026-02-17T12:21:34.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705766 2026-02-17T12:21:34.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836590 1-38654705766' 2026-02-17T12:21:34.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:34.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:34.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574942 2026-02-17T12:21:34.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574942 2026-02-17T12:21:34.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836590 1-38654705766 2-55834574942' 2026-02-17T12:21:34.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:34.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836590 2026-02-17T12:21:34.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:34.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:34.380 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836590 2026-02-17T12:21:34.380 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:34.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836590 2026-02-17T12:21:34.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836590' 2026-02-17T12:21:34.382 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836590 2026-02-17T12:21:34.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:34.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836588 -lt 21474836590 2026-02-17T12:21:34.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:35.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:35.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:35.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836591 -lt 21474836590 2026-02-17T12:21:35.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:35.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705766 2026-02-17T12:21:35.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:35.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:35.703 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705766 2026-02-17T12:21:35.703 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:35.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705766 2026-02-17T12:21:35.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705766' 2026-02-17T12:21:35.704 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705766 2026-02-17T12:21:35.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:35.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705767 -lt 38654705766 2026-02-17T12:21:35.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:35.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574942 2026-02-17T12:21:35.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:35.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:35.864 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574942 2026-02-17T12:21:35.864 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:35.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574942 2026-02-17T12:21:35.866 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574942 2026-02-17T12:21:35.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574942' 2026-02-17T12:21:35.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:36.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574943 -lt 55834574942 2026-02-17T12:21:36.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:36.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:36.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:36.171 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:36.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:36.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:36.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:36.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:36.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:36.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:36.327 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:36.327 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:36.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:36.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:36.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:36.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836593 2026-02-17T12:21:36.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836593 2026-02-17T12:21:36.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836593' 2026-02-17T12:21:36.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:36.398 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:36.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705769 2026-02-17T12:21:36.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705769 2026-02-17T12:21:36.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836593 1-38654705769' 2026-02-17T12:21:36.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:36.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:36.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574945 2026-02-17T12:21:36.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574945 2026-02-17T12:21:36.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836593 1-38654705769 2-55834574945' 2026-02-17T12:21:36.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:36.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836593 2026-02-17T12:21:36.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:36.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:36.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836593 2026-02-17T12:21:36.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:36.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836593 2026-02-17T12:21:36.544 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836593 2026-02-17T12:21:36.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836593' 2026-02-17T12:21:36.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:36.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836591 -lt 21474836593 2026-02-17T12:21:36.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:37.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:37.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:37.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836594 -lt 21474836593 2026-02-17T12:21:37.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:37.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705769 2026-02-17T12:21:37.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:37.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:37.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705769 2026-02-17T12:21:37.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:37.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705769 2026-02-17T12:21:37.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705769' 2026-02-17T12:21:37.857 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705769 2026-02-17T12:21:37.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:38.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705770 -lt 38654705769 2026-02-17T12:21:38.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:38.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574945 2026-02-17T12:21:38.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:38.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:38.008 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574945 2026-02-17T12:21:38.008 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:38.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574945 2026-02-17T12:21:38.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574945' 2026-02-17T12:21:38.010 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574945 2026-02-17T12:21:38.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:38.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574946 -lt 55834574945 2026-02-17T12:21:38.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:38.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:38.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:38.308 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:38.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:38.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:38.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:38.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:38.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:38.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:38.472 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:38.472 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:38.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:38.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:38.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:38.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836596 2026-02-17T12:21:38.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836596 2026-02-17T12:21:38.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836596' 2026-02-17T12:21:38.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:38.541 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:38.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705772 2026-02-17T12:21:38.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705772 2026-02-17T12:21:38.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836596 1-38654705772' 2026-02-17T12:21:38.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:38.614 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:38.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574949 2026-02-17T12:21:38.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574949 2026-02-17T12:21:38.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836596 1-38654705772 2-55834574949' 2026-02-17T12:21:38.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:38.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836596 2026-02-17T12:21:38.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:38.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:38.687 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836596 2026-02-17T12:21:38.687 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:38.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836596 2026-02-17T12:21:38.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836596' 2026-02-17T12:21:38.689 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836596 2026-02-17T12:21:38.689 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:38.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836594 -lt 21474836596 2026-02-17T12:21:38.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:39.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:39.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:40.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836597 -lt 21474836596 2026-02-17T12:21:40.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:40.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705772 2026-02-17T12:21:40.011 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:40.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:40.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705772 2026-02-17T12:21:40.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:40.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705772 2026-02-17T12:21:40.015 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705772 2026-02-17T12:21:40.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705772' 2026-02-17T12:21:40.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:40.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705773 -lt 38654705772 2026-02-17T12:21:40.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:40.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574949 2026-02-17T12:21:40.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:40.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:40.177 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574949 2026-02-17T12:21:40.177 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:40.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574949 2026-02-17T12:21:40.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574949' 2026-02-17T12:21:40.179 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574949 2026-02-17T12:21:40.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:40.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574949 -lt 55834574949 2026-02-17T12:21:40.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:40.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:40.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:40.473 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:40.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:40.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:40.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:40.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:40.483 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:40.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:40.632 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:40.632 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:40.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:40.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:40.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:40.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836599 2026-02-17T12:21:40.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836599 2026-02-17T12:21:40.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836599' 2026-02-17T12:21:40.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:40.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:40.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705776 2026-02-17T12:21:40.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705776 2026-02-17T12:21:40.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836599 1-38654705776' 2026-02-17T12:21:40.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:40.776 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:40.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574952 2026-02-17T12:21:40.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574952 2026-02-17T12:21:40.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836599 1-38654705776 2-55834574952' 2026-02-17T12:21:40.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:40.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836599 2026-02-17T12:21:40.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:40.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:40.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836599 2026-02-17T12:21:40.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:40.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836599 2026-02-17T12:21:40.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836599' 2026-02-17T12:21:40.856 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836599 2026-02-17T12:21:40.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:41.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836597 -lt 21474836599 2026-02-17T12:21:41.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:42.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:42.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:42.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836600 -lt 21474836599 2026-02-17T12:21:42.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:42.167 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705776 2026-02-17T12:21:42.167 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:42.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:42.169 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705776 2026-02-17T12:21:42.169 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:42.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705776 2026-02-17T12:21:42.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705776' 2026-02-17T12:21:42.171 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705776 2026-02-17T12:21:42.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:42.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705776 -lt 38654705776 2026-02-17T12:21:42.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:42.333 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574952 2026-02-17T12:21:42.333 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:42.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:42.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574952 2026-02-17T12:21:42.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:42.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574952 2026-02-17T12:21:42.336 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574952 2026-02-17T12:21:42.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574952' 2026-02-17T12:21:42.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:42.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574952 -lt 55834574952 2026-02-17T12:21:42.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:42.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:42.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:42.648 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:42.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:42.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:42.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:42.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:42.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:42.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:42.806 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:42.806 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:42.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:42.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:42.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:42.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836602 2026-02-17T12:21:42.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836602 2026-02-17T12:21:42.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836602' 2026-02-17T12:21:42.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:42.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:42.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705779 2026-02-17T12:21:42.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705779 2026-02-17T12:21:42.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836602 1-38654705779' 2026-02-17T12:21:42.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:42.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:43.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574955 2026-02-17T12:21:43.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574955 2026-02-17T12:21:43.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836602 1-38654705779 2-55834574955' 2026-02-17T12:21:43.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:43.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836602 2026-02-17T12:21:43.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:43.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:43.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836602 2026-02-17T12:21:43.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:43.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836602 2026-02-17T12:21:43.025 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836602 2026-02-17T12:21:43.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836602' 2026-02-17T12:21:43.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:43.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836600 -lt 21474836602 2026-02-17T12:21:43.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:44.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:44.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:44.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836603 -lt 21474836602 2026-02-17T12:21:44.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:44.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705779 2026-02-17T12:21:44.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:44.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:44.347 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705779 2026-02-17T12:21:44.347 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:44.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705779 2026-02-17T12:21:44.349 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705779 2026-02-17T12:21:44.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705779' 2026-02-17T12:21:44.349 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:44.498 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705779 -lt 38654705779 2026-02-17T12:21:44.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:44.499 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574955 2026-02-17T12:21:44.500 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:44.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:44.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574955 2026-02-17T12:21:44.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:44.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574955 2026-02-17T12:21:44.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574955' 2026-02-17T12:21:44.503 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574955 2026-02-17T12:21:44.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:44.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574955 -lt 55834574955 2026-02-17T12:21:44.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:44.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:44.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:44.812 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:44.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:44.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:44.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:44.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:44.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:44.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:44.975 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:44.975 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:44.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:44.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:44.976 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:45.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836605 2026-02-17T12:21:45.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836605 2026-02-17T12:21:45.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836605' 2026-02-17T12:21:45.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:45.049 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:45.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705782 2026-02-17T12:21:45.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705782 2026-02-17T12:21:45.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836605 1-38654705782' 2026-02-17T12:21:45.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:45.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:45.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574958 2026-02-17T12:21:45.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574958 2026-02-17T12:21:45.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836605 1-38654705782 2-55834574958' 2026-02-17T12:21:45.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:45.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836605 2026-02-17T12:21:45.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:45.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:45.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836605 2026-02-17T12:21:45.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:45.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836605 2026-02-17T12:21:45.198 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836605 2026-02-17T12:21:45.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836605' 2026-02-17T12:21:45.198 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:45.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836603 -lt 21474836605 2026-02-17T12:21:45.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:46.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:46.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:46.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836606 -lt 21474836605 2026-02-17T12:21:46.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:46.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705782 2026-02-17T12:21:46.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:46.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:46.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705782 2026-02-17T12:21:46.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:46.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705782 2026-02-17T12:21:46.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705782' 2026-02-17T12:21:46.517 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705782 2026-02-17T12:21:46.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:46.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705782 -lt 38654705782 2026-02-17T12:21:46.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:46.667 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574958 2026-02-17T12:21:46.667 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:46.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:46.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574958 2026-02-17T12:21:46.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:46.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574958 2026-02-17T12:21:46.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574958' 2026-02-17T12:21:46.671 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574958 2026-02-17T12:21:46.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:46.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574958 -lt 55834574958 2026-02-17T12:21:46.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:46.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:46.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:46.971 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:46.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:46.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:46.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:46.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:46.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:47.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:47.128 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:47.129 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:47.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:47.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:47.129 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:47.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836609 2026-02-17T12:21:47.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836609 2026-02-17T12:21:47.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836609' 2026-02-17T12:21:47.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:47.201 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:47.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705785 2026-02-17T12:21:47.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705785 2026-02-17T12:21:47.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836609 1-38654705785' 2026-02-17T12:21:47.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:47.272 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:47.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574961 2026-02-17T12:21:47.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574961 2026-02-17T12:21:47.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836609 1-38654705785 2-55834574961' 2026-02-17T12:21:47.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:47.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836609 2026-02-17T12:21:47.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:47.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:47.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836609 2026-02-17T12:21:47.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:47.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836609 2026-02-17T12:21:47.344 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836609 2026-02-17T12:21:47.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836609' 2026-02-17T12:21:47.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:47.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836606 -lt 21474836609 2026-02-17T12:21:47.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:48.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:48.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:48.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836609 -lt 21474836609 2026-02-17T12:21:48.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:48.661 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705785 2026-02-17T12:21:48.661 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:48.662 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:48.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705785 2026-02-17T12:21:48.663 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:48.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705785 2026-02-17T12:21:48.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705785' 2026-02-17T12:21:48.664 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705785 2026-02-17T12:21:48.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:48.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705785 -lt 38654705785 2026-02-17T12:21:48.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:48.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574961 2026-02-17T12:21:48.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:48.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:48.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574961 2026-02-17T12:21:48.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:48.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574961 2026-02-17T12:21:48.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574961' 2026-02-17T12:21:48.827 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574961 2026-02-17T12:21:48.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:48.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574961 -lt 55834574961 2026-02-17T12:21:48.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:48.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:48.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:49.127 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:49.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:49.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:49.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:49.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:49.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:49.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:49.284 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:49.284 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:49.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:49.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:49.284 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:49.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836612 2026-02-17T12:21:49.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836612 2026-02-17T12:21:49.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836612' 2026-02-17T12:21:49.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:49.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:49.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705788 2026-02-17T12:21:49.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705788 2026-02-17T12:21:49.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836612 1-38654705788' 2026-02-17T12:21:49.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:49.429 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:49.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574964 2026-02-17T12:21:49.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574964 2026-02-17T12:21:49.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836612 1-38654705788 2-55834574964' 2026-02-17T12:21:49.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:49.502 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836612 2026-02-17T12:21:49.502 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:49.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:49.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836612 2026-02-17T12:21:49.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:49.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836612 2026-02-17T12:21:49.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836612' 2026-02-17T12:21:49.506 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836612 2026-02-17T12:21:49.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:49.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836612 -lt 21474836612 2026-02-17T12:21:49.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:49.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705788 2026-02-17T12:21:49.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:49.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:49.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705788 2026-02-17T12:21:49.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:49.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705788 2026-02-17T12:21:49.675 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705788 2026-02-17T12:21:49.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705788' 2026-02-17T12:21:49.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:49.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705788 -lt 38654705788 2026-02-17T12:21:49.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:49.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574964 2026-02-17T12:21:49.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:49.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:49.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574964 2026-02-17T12:21:49.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:49.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574964 2026-02-17T12:21:49.842 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574964 2026-02-17T12:21:49.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574964' 2026-02-17T12:21:49.843 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:50.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574963 -lt 55834574964 2026-02-17T12:21:50.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:51.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:51.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:51.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574963 -lt 55834574964 2026-02-17T12:21:51.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:52.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:21:52.164 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:52.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574966 -lt 55834574964 2026-02-17T12:21:52.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:52.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:52.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:52.460 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:52.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:52.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:52.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:52.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:52.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:52.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:52.617 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:52.617 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:52.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:52.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:52.617 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:52.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836616 2026-02-17T12:21:52.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836616 2026-02-17T12:21:52.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836616' 2026-02-17T12:21:52.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:52.691 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:52.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705793 2026-02-17T12:21:52.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705793 2026-02-17T12:21:52.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836616 1-38654705793' 2026-02-17T12:21:52.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:52.762 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:52.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574969 2026-02-17T12:21:52.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574969 2026-02-17T12:21:52.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836616 1-38654705793 2-55834574969' 2026-02-17T12:21:52.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:52.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836616 2026-02-17T12:21:52.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:52.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:52.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836616 2026-02-17T12:21:52.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:52.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836616 2026-02-17T12:21:52.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836616' 2026-02-17T12:21:52.837 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836616 2026-02-17T12:21:52.837 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:53.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836614 -lt 21474836616 2026-02-17T12:21:53.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:54.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:54.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:54.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836617 -lt 21474836616 2026-02-17T12:21:54.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:54.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705793 2026-02-17T12:21:54.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:54.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:54.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705793 2026-02-17T12:21:54.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:54.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705793 2026-02-17T12:21:54.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705793' 2026-02-17T12:21:54.159 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705793 2026-02-17T12:21:54.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:54.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705793 -lt 38654705793 2026-02-17T12:21:54.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:54.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574969 2026-02-17T12:21:54.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:54.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:54.329 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574969 2026-02-17T12:21:54.329 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:54.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574969 2026-02-17T12:21:54.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574969' 2026-02-17T12:21:54.331 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574969 2026-02-17T12:21:54.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:54.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574969 -lt 55834574969 2026-02-17T12:21:54.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:54.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:54.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:54.624 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:54.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:54.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:54.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:54.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:54.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:54.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:54.789 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:54.790 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:54.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:54.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:54.790 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:54.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836619 2026-02-17T12:21:54.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836619 2026-02-17T12:21:54.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836619' 2026-02-17T12:21:54.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:54.858 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:54.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705796 2026-02-17T12:21:54.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705796 2026-02-17T12:21:54.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836619 1-38654705796' 2026-02-17T12:21:54.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:54.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:54.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574972 2026-02-17T12:21:55.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574972 2026-02-17T12:21:55.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836619 1-38654705796 2-55834574972' 2026-02-17T12:21:55.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:55.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836619 2026-02-17T12:21:55.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:55.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:55.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836619 2026-02-17T12:21:55.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:55.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836619 2026-02-17T12:21:55.004 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836619 2026-02-17T12:21:55.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836619' 2026-02-17T12:21:55.004 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:55.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836617 -lt 21474836619 2026-02-17T12:21:55.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:56.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:56.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:56.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836620 -lt 21474836619 2026-02-17T12:21:56.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:56.312 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705796 2026-02-17T12:21:56.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:56.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:56.315 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705796 2026-02-17T12:21:56.315 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:56.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705796 2026-02-17T12:21:56.316 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705796 2026-02-17T12:21:56.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705796' 2026-02-17T12:21:56.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:56.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705796 -lt 38654705796 2026-02-17T12:21:56.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:56.476 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574972 2026-02-17T12:21:56.476 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:56.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:56.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574972 2026-02-17T12:21:56.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:56.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574972 2026-02-17T12:21:56.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574972' 2026-02-17T12:21:56.480 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574972 2026-02-17T12:21:56.480 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:56.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574972 -lt 55834574972 2026-02-17T12:21:56.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:56.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:56.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:56.776 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:56.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:56.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:56.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:56.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:56.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:56.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:56.938 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:56.938 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:56.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:56.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:56.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:57.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836622 2026-02-17T12:21:57.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836622 2026-02-17T12:21:57.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836622' 2026-02-17T12:21:57.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:57.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:57.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705799 2026-02-17T12:21:57.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705799 2026-02-17T12:21:57.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836622 1-38654705799' 2026-02-17T12:21:57.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:57.081 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:57.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574975 2026-02-17T12:21:57.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574975 2026-02-17T12:21:57.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836622 1-38654705799 2-55834574975' 2026-02-17T12:21:57.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:57.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836622 2026-02-17T12:21:57.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:57.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:57.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836622 2026-02-17T12:21:57.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:57.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836622 2026-02-17T12:21:57.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836622' 2026-02-17T12:21:57.157 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836622 2026-02-17T12:21:57.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:57.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836620 -lt 21474836622 2026-02-17T12:21:57.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:21:58.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:21:58.311 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:58.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836623 -lt 21474836622 2026-02-17T12:21:58.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:58.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705799 2026-02-17T12:21:58.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:58.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:21:58.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705799 2026-02-17T12:21:58.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:58.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705799 2026-02-17T12:21:58.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705799' 2026-02-17T12:21:58.471 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705799 2026-02-17T12:21:58.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:21:58.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705799 -lt 38654705799 2026-02-17T12:21:58.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:58.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574975 2026-02-17T12:21:58.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:58.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:21:58.627 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574975 2026-02-17T12:21:58.627 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:58.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574975 2026-02-17T12:21:58.628 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574975 2026-02-17T12:21:58.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574975' 2026-02-17T12:21:58.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:21:58.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574975 -lt 55834574975 2026-02-17T12:21:58.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:21:58.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:21:58.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:21:58.929 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:21:58.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:21:58.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:21:58.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:21:58.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:21:58.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:21:59.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:21:59.084 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:21:59.084 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:21:59.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:21:59.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:59.084 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:21:59.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836626 2026-02-17T12:21:59.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836626 2026-02-17T12:21:59.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836626' 2026-02-17T12:21:59.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:59.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:21:59.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705802 2026-02-17T12:21:59.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705802 2026-02-17T12:21:59.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836626 1-38654705802' 2026-02-17T12:21:59.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:21:59.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:21:59.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574978 2026-02-17T12:21:59.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574978 2026-02-17T12:21:59.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836626 1-38654705802 2-55834574978' 2026-02-17T12:21:59.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:21:59.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836626 2026-02-17T12:21:59.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:21:59.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:21:59.303 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836626 2026-02-17T12:21:59.303 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:21:59.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836626 2026-02-17T12:21:59.304 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836626 2026-02-17T12:21:59.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836626' 2026-02-17T12:21:59.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:21:59.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836623 -lt 21474836626 2026-02-17T12:21:59.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:00.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:00.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:00.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836626 -lt 21474836626 2026-02-17T12:22:00.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:00.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705802 2026-02-17T12:22:00.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:00.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:00.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705802 2026-02-17T12:22:00.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:00.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705802 2026-02-17T12:22:00.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705802' 2026-02-17T12:22:00.610 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705802 2026-02-17T12:22:00.610 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:00.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705802 -lt 38654705802 2026-02-17T12:22:00.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:00.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574978 2026-02-17T12:22:00.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:00.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:00.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574978 2026-02-17T12:22:00.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:00.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574978 2026-02-17T12:22:00.779 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574978 2026-02-17T12:22:00.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574978' 2026-02-17T12:22:00.780 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:00.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574978 -lt 55834574978 2026-02-17T12:22:00.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:00.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:00.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:01.084 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:01.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:01.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:01.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:01.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:01.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:01.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:01.251 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:01.251 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:01.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:01.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:01.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:01.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836629 2026-02-17T12:22:01.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836629 2026-02-17T12:22:01.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836629' 2026-02-17T12:22:01.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:01.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:01.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705805 2026-02-17T12:22:01.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705805 2026-02-17T12:22:01.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836629 1-38654705805' 2026-02-17T12:22:01.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:01.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:01.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574981 2026-02-17T12:22:01.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574981 2026-02-17T12:22:01.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836629 1-38654705805 2-55834574981' 2026-02-17T12:22:01.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:01.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836629 2026-02-17T12:22:01.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:01.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:01.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836629 2026-02-17T12:22:01.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:01.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836629 2026-02-17T12:22:01.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836629' 2026-02-17T12:22:01.469 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836629 2026-02-17T12:22:01.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:01.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836629 -lt 21474836629 2026-02-17T12:22:01.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:01.627 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705805 2026-02-17T12:22:01.627 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:01.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:01.629 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705805 2026-02-17T12:22:01.629 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:01.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705805 2026-02-17T12:22:01.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705805' 2026-02-17T12:22:01.630 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705805 2026-02-17T12:22:01.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:01.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705805 -lt 38654705805 2026-02-17T12:22:01.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:01.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574981 2026-02-17T12:22:01.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:01.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:01.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574981 2026-02-17T12:22:01.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:01.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574981 2026-02-17T12:22:01.798 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574981 2026-02-17T12:22:01.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574981' 2026-02-17T12:22:01.798 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:01.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574981 -lt 55834574981 2026-02-17T12:22:01.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:01.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:01.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:02.098 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:02.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:02.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:02.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:02.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:02.110 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:02.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:02.265 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:02.265 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:02.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:02.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:02.266 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:02.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836631 2026-02-17T12:22:02.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836631 2026-02-17T12:22:02.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836631' 2026-02-17T12:22:02.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:02.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:02.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705807 2026-02-17T12:22:02.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705807 2026-02-17T12:22:02.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836631 1-38654705807' 2026-02-17T12:22:02.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:02.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:02.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574983 2026-02-17T12:22:02.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574983 2026-02-17T12:22:02.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836631 1-38654705807 2-55834574983' 2026-02-17T12:22:02.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:02.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836631 2026-02-17T12:22:02.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:02.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:02.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836631 2026-02-17T12:22:02.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:02.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836631 2026-02-17T12:22:02.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836631' 2026-02-17T12:22:02.480 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836631 2026-02-17T12:22:02.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:02.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836629 -lt 21474836631 2026-02-17T12:22:02.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:03.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:03.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:03.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836632 -lt 21474836631 2026-02-17T12:22:03.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:03.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705807 2026-02-17T12:22:03.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:03.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:03.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705807 2026-02-17T12:22:03.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:03.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705807 2026-02-17T12:22:03.791 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705807 2026-02-17T12:22:03.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705807' 2026-02-17T12:22:03.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:03.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705808 -lt 38654705807 2026-02-17T12:22:03.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:03.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574983 2026-02-17T12:22:03.956 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:03.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:03.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574983 2026-02-17T12:22:03.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:03.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574983 2026-02-17T12:22:03.959 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574983 2026-02-17T12:22:03.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574983' 2026-02-17T12:22:03.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:04.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574984 -lt 55834574983 2026-02-17T12:22:04.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:04.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:04.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:04.259 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:04.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:04.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:04.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:04.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:04.269 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:04.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:04.422 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:04.422 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:04.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:04.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:04.423 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:04.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836634 2026-02-17T12:22:04.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836634 2026-02-17T12:22:04.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836634' 2026-02-17T12:22:04.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:04.492 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:04.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705810 2026-02-17T12:22:04.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705810 2026-02-17T12:22:04.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836634 1-38654705810' 2026-02-17T12:22:04.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:04.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:04.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574987 2026-02-17T12:22:04.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574987 2026-02-17T12:22:04.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836634 1-38654705810 2-55834574987' 2026-02-17T12:22:04.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:04.636 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836634 2026-02-17T12:22:04.636 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:04.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:04.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836634 2026-02-17T12:22:04.638 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:04.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836634 2026-02-17T12:22:04.640 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836634 2026-02-17T12:22:04.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836634' 2026-02-17T12:22:04.640 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:04.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836632 -lt 21474836634 2026-02-17T12:22:04.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:05.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:05.798 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:05.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836635 -lt 21474836634 2026-02-17T12:22:05.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:05.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705810 2026-02-17T12:22:05.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:05.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:05.954 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705810 2026-02-17T12:22:05.954 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:05.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705810 2026-02-17T12:22:05.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705810' 2026-02-17T12:22:05.955 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705810 2026-02-17T12:22:05.956 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:06.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705811 -lt 38654705810 2026-02-17T12:22:06.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:06.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574987 2026-02-17T12:22:06.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:06.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:06.114 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574987 2026-02-17T12:22:06.114 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:06.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574987 2026-02-17T12:22:06.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574987' 2026-02-17T12:22:06.116 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574987 2026-02-17T12:22:06.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:06.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574987 -lt 55834574987 2026-02-17T12:22:06.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:06.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:06.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:06.408 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:06.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:06.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:06.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:06.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:06.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:06.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:06.570 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:06.570 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:06.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:06.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:06.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:06.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836637 2026-02-17T12:22:06.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836637 2026-02-17T12:22:06.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836637' 2026-02-17T12:22:06.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:06.640 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:06.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705814 2026-02-17T12:22:06.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705814 2026-02-17T12:22:06.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836637 1-38654705814' 2026-02-17T12:22:06.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:06.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:06.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574990 2026-02-17T12:22:06.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574990 2026-02-17T12:22:06.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836637 1-38654705814 2-55834574990' 2026-02-17T12:22:06.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:06.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836637 2026-02-17T12:22:06.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:06.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:06.790 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836637 2026-02-17T12:22:06.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:06.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836637 2026-02-17T12:22:06.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836637' 2026-02-17T12:22:06.792 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836637 2026-02-17T12:22:06.793 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:06.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836635 -lt 21474836637 2026-02-17T12:22:06.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:07.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:07.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:08.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836638 -lt 21474836637 2026-02-17T12:22:08.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:08.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705814 2026-02-17T12:22:08.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:08.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:08.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705814 2026-02-17T12:22:08.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:08.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705814 2026-02-17T12:22:08.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705814' 2026-02-17T12:22:08.102 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705814 2026-02-17T12:22:08.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:08.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705814 -lt 38654705814 2026-02-17T12:22:08.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:08.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574990 2026-02-17T12:22:08.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:08.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:08.267 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574990 2026-02-17T12:22:08.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:08.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574990 2026-02-17T12:22:08.269 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574990 2026-02-17T12:22:08.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574990' 2026-02-17T12:22:08.270 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:08.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574990 -lt 55834574990 2026-02-17T12:22:08.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:08.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:08.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:08.573 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:08.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:08.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:08.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:08.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:08.583 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:08.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:08.730 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:08.730 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:08.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:08.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:08.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:08.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836640 2026-02-17T12:22:08.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836640 2026-02-17T12:22:08.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836640' 2026-02-17T12:22:08.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:08.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:08.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705817 2026-02-17T12:22:08.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705817 2026-02-17T12:22:08.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836640 1-38654705817' 2026-02-17T12:22:08.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:08.873 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:08.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574993 2026-02-17T12:22:08.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574993 2026-02-17T12:22:08.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836640 1-38654705817 2-55834574993' 2026-02-17T12:22:08.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:08.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836640 2026-02-17T12:22:08.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:08.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:08.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836640 2026-02-17T12:22:08.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:08.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836640 2026-02-17T12:22:08.948 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836640 2026-02-17T12:22:08.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836640' 2026-02-17T12:22:08.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:09.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836638 -lt 21474836640 2026-02-17T12:22:09.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:10.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:10.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:10.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836641 -lt 21474836640 2026-02-17T12:22:10.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:10.269 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705817 2026-02-17T12:22:10.269 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:10.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:10.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705817 2026-02-17T12:22:10.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:10.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705817 2026-02-17T12:22:10.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705817' 2026-02-17T12:22:10.273 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705817 2026-02-17T12:22:10.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:10.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705817 -lt 38654705817 2026-02-17T12:22:10.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:10.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574993 2026-02-17T12:22:10.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:10.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:10.437 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574993 2026-02-17T12:22:10.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:10.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574993 2026-02-17T12:22:10.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574993' 2026-02-17T12:22:10.439 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574993 2026-02-17T12:22:10.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:10.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574993 -lt 55834574993 2026-02-17T12:22:10.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:10.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:10.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:10.728 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:10.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:10.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:10.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:10.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:10.740 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:10.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:10.889 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:10.889 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:10.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:10.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:10.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:10.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836643 2026-02-17T12:22:10.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836643 2026-02-17T12:22:10.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836643' 2026-02-17T12:22:10.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:10.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:11.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705820 2026-02-17T12:22:11.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705820 2026-02-17T12:22:11.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836643 1-38654705820' 2026-02-17T12:22:11.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:11.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:11.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574996 2026-02-17T12:22:11.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574996 2026-02-17T12:22:11.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836643 1-38654705820 2-55834574996' 2026-02-17T12:22:11.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:11.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836643 2026-02-17T12:22:11.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:11.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:11.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836643 2026-02-17T12:22:11.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:11.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836643 2026-02-17T12:22:11.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836643' 2026-02-17T12:22:11.106 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836643 2026-02-17T12:22:11.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:11.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836641 -lt 21474836643 2026-02-17T12:22:11.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:12.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:12.257 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:12.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836644 -lt 21474836643 2026-02-17T12:22:12.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:12.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705820 2026-02-17T12:22:12.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:12.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:12.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705820 2026-02-17T12:22:12.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:12.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705820 2026-02-17T12:22:12.418 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705820 2026-02-17T12:22:12.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705820' 2026-02-17T12:22:12.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:12.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705820 -lt 38654705820 2026-02-17T12:22:12.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:12.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574996 2026-02-17T12:22:12.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:12.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:12.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574996 2026-02-17T12:22:12.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:12.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574996 2026-02-17T12:22:12.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574996' 2026-02-17T12:22:12.580 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574996 2026-02-17T12:22:12.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:12.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574996 -lt 55834574996 2026-02-17T12:22:12.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:12.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:12.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:12.879 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:12.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:12.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:12.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:12.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:12.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:13.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:13.043 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:13.043 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:13.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:13.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:13.043 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:13.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836646 2026-02-17T12:22:13.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836646 2026-02-17T12:22:13.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836646' 2026-02-17T12:22:13.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:13.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:13.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705823 2026-02-17T12:22:13.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705823 2026-02-17T12:22:13.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836646 1-38654705823' 2026-02-17T12:22:13.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:13.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:13.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574999 2026-02-17T12:22:13.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574999 2026-02-17T12:22:13.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836646 1-38654705823 2-55834574999' 2026-02-17T12:22:13.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:13.258 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836646 2026-02-17T12:22:13.258 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:13.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:13.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836646 2026-02-17T12:22:13.260 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:13.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836646 2026-02-17T12:22:13.261 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836646 2026-02-17T12:22:13.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836646' 2026-02-17T12:22:13.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:13.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836644 -lt 21474836646 2026-02-17T12:22:13.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:14.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:14.421 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:14.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836647 -lt 21474836646 2026-02-17T12:22:14.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:14.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705823 2026-02-17T12:22:14.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:14.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:14.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705823 2026-02-17T12:22:14.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:14.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705823 2026-02-17T12:22:14.581 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705823 2026-02-17T12:22:14.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705823' 2026-02-17T12:22:14.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:14.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705823 -lt 38654705823 2026-02-17T12:22:14.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:14.740 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574999 2026-02-17T12:22:14.741 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:14.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:14.742 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574999 2026-02-17T12:22:14.743 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:14.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574999 2026-02-17T12:22:14.744 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574999 2026-02-17T12:22:14.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574999' 2026-02-17T12:22:14.745 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:14.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574999 -lt 55834574999 2026-02-17T12:22:14.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:14.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:14.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:15.042 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:15.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:15.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:15.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:15.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:15.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:15.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:15.195 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:15.195 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:15.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:15.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:15.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:15.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836650 2026-02-17T12:22:15.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836650 2026-02-17T12:22:15.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836650' 2026-02-17T12:22:15.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:15.266 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:15.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705826 2026-02-17T12:22:15.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705826 2026-02-17T12:22:15.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836650 1-38654705826' 2026-02-17T12:22:15.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:15.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:15.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575002 2026-02-17T12:22:15.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575002 2026-02-17T12:22:15.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836650 1-38654705826 2-55834575002' 2026-02-17T12:22:15.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:15.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836650 2026-02-17T12:22:15.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:15.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:15.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836650 2026-02-17T12:22:15.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:15.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836650 2026-02-17T12:22:15.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836650' 2026-02-17T12:22:15.414 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836650 2026-02-17T12:22:15.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:15.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836650 -lt 21474836650 2026-02-17T12:22:15.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:15.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705826 2026-02-17T12:22:15.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:15.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:15.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705826 2026-02-17T12:22:15.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:15.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705826 2026-02-17T12:22:15.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705826' 2026-02-17T12:22:15.579 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705826 2026-02-17T12:22:15.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:15.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705826 -lt 38654705826 2026-02-17T12:22:15.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:15.745 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575002 2026-02-17T12:22:15.745 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:15.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:15.747 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575002 2026-02-17T12:22:15.747 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:15.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575002 2026-02-17T12:22:15.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575002' 2026-02-17T12:22:15.748 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575002 2026-02-17T12:22:15.749 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:15.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575002 -lt 55834575002 2026-02-17T12:22:15.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:15.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:15.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:16.051 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:16.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:16.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:16.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:16.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:16.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:16.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:16.206 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:16.206 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:16.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:16.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:16.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:16.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836652 2026-02-17T12:22:16.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836652 2026-02-17T12:22:16.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836652' 2026-02-17T12:22:16.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:16.278 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:16.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705828 2026-02-17T12:22:16.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705828 2026-02-17T12:22:16.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836652 1-38654705828' 2026-02-17T12:22:16.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:16.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:16.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575004 2026-02-17T12:22:16.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575004 2026-02-17T12:22:16.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836652 1-38654705828 2-55834575004' 2026-02-17T12:22:16.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:16.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836652 2026-02-17T12:22:16.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:16.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:16.423 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836652 2026-02-17T12:22:16.423 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:16.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836652 2026-02-17T12:22:16.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836652' 2026-02-17T12:22:16.424 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836652 2026-02-17T12:22:16.425 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:16.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836650 -lt 21474836652 2026-02-17T12:22:16.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:17.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:17.574 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:17.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836653 -lt 21474836652 2026-02-17T12:22:17.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:17.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705828 2026-02-17T12:22:17.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:17.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:17.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705828 2026-02-17T12:22:17.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:17.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705828 2026-02-17T12:22:17.734 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705828 2026-02-17T12:22:17.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705828' 2026-02-17T12:22:17.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:17.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705829 -lt 38654705828 2026-02-17T12:22:17.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:17.896 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575004 2026-02-17T12:22:17.896 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:17.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:17.898 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575004 2026-02-17T12:22:17.898 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:17.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575004 2026-02-17T12:22:17.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575004' 2026-02-17T12:22:17.900 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575004 2026-02-17T12:22:17.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:18.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575005 -lt 55834575004 2026-02-17T12:22:18.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:18.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:18.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:18.203 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:18.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:18.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:18.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:18.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:18.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:18.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:18.358 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:18.358 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:18.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:18.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:18.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:18.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836655 2026-02-17T12:22:18.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836655 2026-02-17T12:22:18.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836655' 2026-02-17T12:22:18.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:18.429 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:18.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705831 2026-02-17T12:22:18.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705831 2026-02-17T12:22:18.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836655 1-38654705831' 2026-02-17T12:22:18.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:18.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:18.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575007 2026-02-17T12:22:18.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575007 2026-02-17T12:22:18.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836655 1-38654705831 2-55834575007' 2026-02-17T12:22:18.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:18.573 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836655 2026-02-17T12:22:18.574 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:18.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:18.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836655 2026-02-17T12:22:18.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:18.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836655 2026-02-17T12:22:18.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836655' 2026-02-17T12:22:18.577 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836655 2026-02-17T12:22:18.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:18.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836653 -lt 21474836655 2026-02-17T12:22:18.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:19.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:19.737 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:19.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836656 -lt 21474836655 2026-02-17T12:22:19.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:19.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705831 2026-02-17T12:22:19.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:19.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:19.896 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705831 2026-02-17T12:22:19.896 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:19.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705831 2026-02-17T12:22:19.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705831' 2026-02-17T12:22:19.898 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705831 2026-02-17T12:22:19.898 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:20.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705832 -lt 38654705831 2026-02-17T12:22:20.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:20.058 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575007 2026-02-17T12:22:20.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:20.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:20.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575007 2026-02-17T12:22:20.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:20.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575007 2026-02-17T12:22:20.062 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575007 2026-02-17T12:22:20.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575007' 2026-02-17T12:22:20.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:20.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575008 -lt 55834575007 2026-02-17T12:22:20.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:20.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:20.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:20.364 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:20.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:20.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:20.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:20.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:20.374 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:20.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:20.519 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:20.519 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:20.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:20.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:20.520 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:20.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836658 2026-02-17T12:22:20.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836658 2026-02-17T12:22:20.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836658' 2026-02-17T12:22:20.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:20.591 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:20.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705834 2026-02-17T12:22:20.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705834 2026-02-17T12:22:20.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836658 1-38654705834' 2026-02-17T12:22:20.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:20.661 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:20.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575011 2026-02-17T12:22:20.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575011 2026-02-17T12:22:20.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836658 1-38654705834 2-55834575011' 2026-02-17T12:22:20.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:20.732 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836658 2026-02-17T12:22:20.732 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:20.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:20.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836658 2026-02-17T12:22:20.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:20.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836658 2026-02-17T12:22:20.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836658' 2026-02-17T12:22:20.736 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836658 2026-02-17T12:22:20.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:20.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836656 -lt 21474836658 2026-02-17T12:22:20.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:21.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:21.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:22.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836659 -lt 21474836658 2026-02-17T12:22:22.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:22.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705834 2026-02-17T12:22:22.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:22.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:22.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705834 2026-02-17T12:22:22.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:22.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705834 2026-02-17T12:22:22.062 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705834 2026-02-17T12:22:22.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705834' 2026-02-17T12:22:22.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:22.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705835 -lt 38654705834 2026-02-17T12:22:22.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:22.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575011 2026-02-17T12:22:22.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:22.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:22.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575011 2026-02-17T12:22:22.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:22.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575011 2026-02-17T12:22:22.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575011' 2026-02-17T12:22:22.230 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575011 2026-02-17T12:22:22.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:22.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575011 -lt 55834575011 2026-02-17T12:22:22.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:22.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:22.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:22.527 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:22.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:22.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:22.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:22.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:22.536 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:22.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:22.684 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:22.684 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:22.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:22.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:22.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:22.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836661 2026-02-17T12:22:22.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836661 2026-02-17T12:22:22.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836661' 2026-02-17T12:22:22.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:22.757 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:22.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705838 2026-02-17T12:22:22.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705838 2026-02-17T12:22:22.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836661 1-38654705838' 2026-02-17T12:22:22.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:22.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:22.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575014 2026-02-17T12:22:22.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575014 2026-02-17T12:22:22.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836661 1-38654705838 2-55834575014' 2026-02-17T12:22:22.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:22.897 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836661 2026-02-17T12:22:22.897 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:22.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:22.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836661 2026-02-17T12:22:22.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:22.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836661 2026-02-17T12:22:22.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836661' 2026-02-17T12:22:22.901 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836661 2026-02-17T12:22:22.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:23.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836659 -lt 21474836661 2026-02-17T12:22:23.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:24.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:24.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:24.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836662 -lt 21474836661 2026-02-17T12:22:24.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:24.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705838 2026-02-17T12:22:24.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:24.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:24.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705838 2026-02-17T12:22:24.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:24.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705838 2026-02-17T12:22:24.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705838' 2026-02-17T12:22:24.214 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705838 2026-02-17T12:22:24.214 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:24.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705838 -lt 38654705838 2026-02-17T12:22:24.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:24.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575014 2026-02-17T12:22:24.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:24.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:24.377 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575014 2026-02-17T12:22:24.377 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:24.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575014 2026-02-17T12:22:24.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575014' 2026-02-17T12:22:24.379 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575014 2026-02-17T12:22:24.379 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:24.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575014 -lt 55834575014 2026-02-17T12:22:24.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:24.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:24.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:24.684 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:24.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:24.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:24.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:24.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:24.694 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:24.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:24.840 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:24.840 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:24.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:24.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:24.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:24.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836664 2026-02-17T12:22:24.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836664 2026-02-17T12:22:24.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836664' 2026-02-17T12:22:24.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:24.913 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:24.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705841 2026-02-17T12:22:24.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705841 2026-02-17T12:22:24.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836664 1-38654705841' 2026-02-17T12:22:24.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:24.986 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:25.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575017 2026-02-17T12:22:25.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575017 2026-02-17T12:22:25.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836664 1-38654705841 2-55834575017' 2026-02-17T12:22:25.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:25.058 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836664 2026-02-17T12:22:25.058 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:25.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:25.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836664 2026-02-17T12:22:25.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:25.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836664 2026-02-17T12:22:25.061 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836664 2026-02-17T12:22:25.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836664' 2026-02-17T12:22:25.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:25.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836662 -lt 21474836664 2026-02-17T12:22:25.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:26.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:26.218 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:26.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836665 -lt 21474836664 2026-02-17T12:22:26.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:26.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705841 2026-02-17T12:22:26.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:26.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:26.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705841 2026-02-17T12:22:26.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:26.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705841 2026-02-17T12:22:26.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705841' 2026-02-17T12:22:26.372 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705841 2026-02-17T12:22:26.372 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:26.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705841 -lt 38654705841 2026-02-17T12:22:26.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:26.531 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575017 2026-02-17T12:22:26.531 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:26.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:26.533 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575017 2026-02-17T12:22:26.533 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:26.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575017 2026-02-17T12:22:26.534 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575017 2026-02-17T12:22:26.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575017' 2026-02-17T12:22:26.535 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:26.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575017 -lt 55834575017 2026-02-17T12:22:26.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:26.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:26.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:26.837 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:26.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:26.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:26.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:26.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:26.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:26.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:26.999 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:27.000 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:27.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:27.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:27.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:27.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836667 2026-02-17T12:22:27.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836667 2026-02-17T12:22:27.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836667' 2026-02-17T12:22:27.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:27.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:27.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705844 2026-02-17T12:22:27.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705844 2026-02-17T12:22:27.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836667 1-38654705844' 2026-02-17T12:22:27.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:27.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:27.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575020 2026-02-17T12:22:27.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575020 2026-02-17T12:22:27.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836667 1-38654705844 2-55834575020' 2026-02-17T12:22:27.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:27.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836667 2026-02-17T12:22:27.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:27.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:27.214 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836667 2026-02-17T12:22:27.214 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:27.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836667 2026-02-17T12:22:27.216 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836667 2026-02-17T12:22:27.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836667' 2026-02-17T12:22:27.216 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:27.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836665 -lt 21474836667 2026-02-17T12:22:27.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:28.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:28.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:28.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836668 -lt 21474836667 2026-02-17T12:22:28.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:28.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705844 2026-02-17T12:22:28.531 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:28.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:28.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705844 2026-02-17T12:22:28.533 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:28.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705844 2026-02-17T12:22:28.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705844' 2026-02-17T12:22:28.534 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705844 2026-02-17T12:22:28.535 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:28.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705844 -lt 38654705844 2026-02-17T12:22:28.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:28.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575020 2026-02-17T12:22:28.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:28.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:28.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575020 2026-02-17T12:22:28.701 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:28.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575020 2026-02-17T12:22:28.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575020' 2026-02-17T12:22:28.702 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575020 2026-02-17T12:22:28.703 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:28.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575020 -lt 55834575020 2026-02-17T12:22:28.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:28.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:28.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:29.000 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:29.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:29.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:29.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:29.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:29.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:29.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:29.158 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:29.158 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:29.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:29.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:29.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:29.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836671 2026-02-17T12:22:29.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836671 2026-02-17T12:22:29.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836671' 2026-02-17T12:22:29.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:29.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:29.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705847 2026-02-17T12:22:29.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705847 2026-02-17T12:22:29.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836671 1-38654705847' 2026-02-17T12:22:29.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:29.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:29.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575023 2026-02-17T12:22:29.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575023 2026-02-17T12:22:29.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836671 1-38654705847 2-55834575023' 2026-02-17T12:22:29.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:29.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836671 2026-02-17T12:22:29.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:29.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:29.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836671 2026-02-17T12:22:29.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:29.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836671 2026-02-17T12:22:29.372 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836671 2026-02-17T12:22:29.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836671' 2026-02-17T12:22:29.373 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:29.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836668 -lt 21474836671 2026-02-17T12:22:29.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:30.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:30.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:30.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836671 -lt 21474836671 2026-02-17T12:22:30.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:30.681 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705847 2026-02-17T12:22:30.681 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:30.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:30.683 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705847 2026-02-17T12:22:30.683 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:30.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705847 2026-02-17T12:22:30.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705847' 2026-02-17T12:22:30.685 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705847 2026-02-17T12:22:30.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:30.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705847 -lt 38654705847 2026-02-17T12:22:30.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:30.851 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575023 2026-02-17T12:22:30.851 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:30.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:30.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575023 2026-02-17T12:22:30.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:30.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575023 2026-02-17T12:22:30.854 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575023 2026-02-17T12:22:30.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575023' 2026-02-17T12:22:30.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:31.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575023 -lt 55834575023 2026-02-17T12:22:31.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:31.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:31.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:31.154 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:31.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:31.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:31.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:31.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:31.164 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:31.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:31.312 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:31.312 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:31.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:31.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:31.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:31.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836674 2026-02-17T12:22:31.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836674 2026-02-17T12:22:31.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836674' 2026-02-17T12:22:31.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:31.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:31.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705850 2026-02-17T12:22:31.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705850 2026-02-17T12:22:31.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836674 1-38654705850' 2026-02-17T12:22:31.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:31.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:31.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575026 2026-02-17T12:22:31.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575026 2026-02-17T12:22:31.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836674 1-38654705850 2-55834575026' 2026-02-17T12:22:31.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:31.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836674 2026-02-17T12:22:31.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:31.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:31.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836674 2026-02-17T12:22:31.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:31.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836674 2026-02-17T12:22:31.530 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836674 2026-02-17T12:22:31.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836674' 2026-02-17T12:22:31.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:31.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836674 -lt 21474836674 2026-02-17T12:22:31.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:31.691 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705850 2026-02-17T12:22:31.691 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:31.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:31.693 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705850 2026-02-17T12:22:31.693 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:31.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705850 2026-02-17T12:22:31.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705850' 2026-02-17T12:22:31.695 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705850 2026-02-17T12:22:31.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:31.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705850 -lt 38654705850 2026-02-17T12:22:31.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:31.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575026 2026-02-17T12:22:31.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:31.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:31.858 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575026 2026-02-17T12:22:31.858 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:31.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575026 2026-02-17T12:22:31.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575026' 2026-02-17T12:22:31.859 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575026 2026-02-17T12:22:31.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:32.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575025 -lt 55834575026 2026-02-17T12:22:32.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:33.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:33.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:33.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575025 -lt 55834575026 2026-02-17T12:22:33.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:34.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:22:34.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:34.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575028 -lt 55834575026 2026-02-17T12:22:34.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:34.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:34.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:34.479 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:34.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:34.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:34.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:34.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:34.489 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:34.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:34.642 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:34.642 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:34.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:34.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:34.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:34.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836678 2026-02-17T12:22:34.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836678 2026-02-17T12:22:34.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836678' 2026-02-17T12:22:34.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:34.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:34.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705855 2026-02-17T12:22:34.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705855 2026-02-17T12:22:34.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836678 1-38654705855' 2026-02-17T12:22:34.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:34.784 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:34.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575031 2026-02-17T12:22:34.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575031 2026-02-17T12:22:34.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836678 1-38654705855 2-55834575031' 2026-02-17T12:22:34.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:34.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836678 2026-02-17T12:22:34.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:34.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:34.858 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836678 2026-02-17T12:22:34.858 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:34.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836678 2026-02-17T12:22:34.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836678' 2026-02-17T12:22:34.860 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836678 2026-02-17T12:22:34.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:35.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836676 -lt 21474836678 2026-02-17T12:22:35.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:36.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:36.019 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:36.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836679 -lt 21474836678 2026-02-17T12:22:36.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:36.170 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705855 2026-02-17T12:22:36.170 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:36.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:36.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705855 2026-02-17T12:22:36.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:36.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705855 2026-02-17T12:22:36.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705855' 2026-02-17T12:22:36.174 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705855 2026-02-17T12:22:36.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:36.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705855 -lt 38654705855 2026-02-17T12:22:36.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:36.338 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575031 2026-02-17T12:22:36.338 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:36.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:36.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575031 2026-02-17T12:22:36.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:36.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575031 2026-02-17T12:22:36.341 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575031 2026-02-17T12:22:36.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575031' 2026-02-17T12:22:36.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:36.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575031 -lt 55834575031 2026-02-17T12:22:36.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:36.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:36.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:36.641 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:36.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:36.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:36.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:36.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:36.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:36.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:36.805 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:36.805 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:36.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:36.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:36.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:36.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836681 2026-02-17T12:22:36.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836681 2026-02-17T12:22:36.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836681' 2026-02-17T12:22:36.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:36.880 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:36.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705858 2026-02-17T12:22:36.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705858 2026-02-17T12:22:36.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836681 1-38654705858' 2026-02-17T12:22:36.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:36.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:37.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575034 2026-02-17T12:22:37.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575034 2026-02-17T12:22:37.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836681 1-38654705858 2-55834575034' 2026-02-17T12:22:37.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:37.021 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836681 2026-02-17T12:22:37.021 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:37.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:37.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836681 2026-02-17T12:22:37.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:37.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836681 2026-02-17T12:22:37.025 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836681 2026-02-17T12:22:37.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836681' 2026-02-17T12:22:37.025 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:37.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836679 -lt 21474836681 2026-02-17T12:22:37.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:38.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:38.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:38.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836682 -lt 21474836681 2026-02-17T12:22:38.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:38.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705858 2026-02-17T12:22:38.345 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:38.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:38.347 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705858 2026-02-17T12:22:38.347 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:38.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705858 2026-02-17T12:22:38.349 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705858 2026-02-17T12:22:38.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705858' 2026-02-17T12:22:38.350 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:38.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705858 -lt 38654705858 2026-02-17T12:22:38.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:38.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575034 2026-02-17T12:22:38.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:38.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:38.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575034 2026-02-17T12:22:38.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:38.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575034 2026-02-17T12:22:38.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575034' 2026-02-17T12:22:38.518 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575034 2026-02-17T12:22:38.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:38.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575034 -lt 55834575034 2026-02-17T12:22:38.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:38.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:38.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:38.813 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:38.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:38.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:38.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:38.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:38.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:38.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:38.977 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:38.977 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:38.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:38.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:38.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:39.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836684 2026-02-17T12:22:39.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836684 2026-02-17T12:22:39.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836684' 2026-02-17T12:22:39.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:39.049 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:39.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705861 2026-02-17T12:22:39.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705861 2026-02-17T12:22:39.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836684 1-38654705861' 2026-02-17T12:22:39.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:39.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:39.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575037 2026-02-17T12:22:39.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575037 2026-02-17T12:22:39.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836684 1-38654705861 2-55834575037' 2026-02-17T12:22:39.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:39.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836684 2026-02-17T12:22:39.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:39.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:39.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836684 2026-02-17T12:22:39.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:39.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836684 2026-02-17T12:22:39.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836684' 2026-02-17T12:22:39.201 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836684 2026-02-17T12:22:39.201 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:39.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836682 -lt 21474836684 2026-02-17T12:22:39.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:40.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:40.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:40.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836685 -lt 21474836684 2026-02-17T12:22:40.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:40.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705861 2026-02-17T12:22:40.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:40.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:40.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705861 2026-02-17T12:22:40.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:40.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705861 2026-02-17T12:22:40.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705861' 2026-02-17T12:22:40.519 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705861 2026-02-17T12:22:40.519 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:40.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705861 -lt 38654705861 2026-02-17T12:22:40.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:40.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575037 2026-02-17T12:22:40.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:40.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:40.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575037 2026-02-17T12:22:40.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:40.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575037 2026-02-17T12:22:40.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575037' 2026-02-17T12:22:40.677 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575037 2026-02-17T12:22:40.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:40.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575037 -lt 55834575037 2026-02-17T12:22:40.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:40.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:40.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:40.978 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:40.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:40.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:40.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:40.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:40.987 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:41.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:41.134 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:41.134 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:41.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:41.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:41.135 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:41.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836688 2026-02-17T12:22:41.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836688 2026-02-17T12:22:41.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836688' 2026-02-17T12:22:41.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:41.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:41.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705864 2026-02-17T12:22:41.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705864 2026-02-17T12:22:41.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836688 1-38654705864' 2026-02-17T12:22:41.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:41.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:41.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575040 2026-02-17T12:22:41.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575040 2026-02-17T12:22:41.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836688 1-38654705864 2-55834575040' 2026-02-17T12:22:41.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:41.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836688 2026-02-17T12:22:41.346 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:41.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:41.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836688 2026-02-17T12:22:41.348 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:41.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836688 2026-02-17T12:22:41.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836688' 2026-02-17T12:22:41.349 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836688 2026-02-17T12:22:41.350 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836685 -lt 21474836688 2026-02-17T12:22:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:42.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:42.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:42.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836688 -lt 21474836688 2026-02-17T12:22:42.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:42.664 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705864 2026-02-17T12:22:42.664 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:42.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:42.667 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705864 2026-02-17T12:22:42.667 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:42.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705864 2026-02-17T12:22:42.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705864' 2026-02-17T12:22:42.668 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705864 2026-02-17T12:22:42.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:42.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705864 -lt 38654705864 2026-02-17T12:22:42.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:42.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575040 2026-02-17T12:22:42.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:42.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:42.834 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575040 2026-02-17T12:22:42.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:42.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575040 2026-02-17T12:22:42.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575040' 2026-02-17T12:22:42.836 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575040 2026-02-17T12:22:42.836 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:42.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575040 -lt 55834575040 2026-02-17T12:22:42.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:42.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:42.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:43.130 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:43.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:43.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:43.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:43.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:43.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:43.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:43.293 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:43.294 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:43.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:43.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:43.294 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:43.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836691 2026-02-17T12:22:43.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836691 2026-02-17T12:22:43.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836691' 2026-02-17T12:22:43.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:43.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:43.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705867 2026-02-17T12:22:43.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705867 2026-02-17T12:22:43.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836691 1-38654705867' 2026-02-17T12:22:43.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:43.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:43.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575043 2026-02-17T12:22:43.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575043 2026-02-17T12:22:43.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836691 1-38654705867 2-55834575043' 2026-02-17T12:22:43.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:43.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836691 2026-02-17T12:22:43.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:43.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:43.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836691 2026-02-17T12:22:43.508 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:43.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836691 2026-02-17T12:22:43.509 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836691 2026-02-17T12:22:43.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836691' 2026-02-17T12:22:43.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:43.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836691 -lt 21474836691 2026-02-17T12:22:43.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:43.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705867 2026-02-17T12:22:43.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:43.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:43.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705867 2026-02-17T12:22:43.676 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:43.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705867 2026-02-17T12:22:43.677 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705867 2026-02-17T12:22:43.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705867' 2026-02-17T12:22:43.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:43.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705867 -lt 38654705867 2026-02-17T12:22:43.838 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:43.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575043 2026-02-17T12:22:43.839 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:43.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:43.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575043 2026-02-17T12:22:43.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:43.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575043 2026-02-17T12:22:43.842 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575043 2026-02-17T12:22:43.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575043' 2026-02-17T12:22:43.843 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:44.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575042 -lt 55834575043 2026-02-17T12:22:44.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:45.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:45.012 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:45.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575042 -lt 55834575043 2026-02-17T12:22:45.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:46.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:22:46.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:46.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575045 -lt 55834575043 2026-02-17T12:22:46.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:46.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:46.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:46.462 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:46.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:46.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:46.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:46.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:46.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:46.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:46.619 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:46.619 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:46.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:46.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:46.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:46.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836695 2026-02-17T12:22:46.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836695 2026-02-17T12:22:46.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836695' 2026-02-17T12:22:46.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:46.691 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:46.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705872 2026-02-17T12:22:46.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705872 2026-02-17T12:22:46.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836695 1-38654705872' 2026-02-17T12:22:46.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:46.761 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:46.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575048 2026-02-17T12:22:46.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575048 2026-02-17T12:22:46.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836695 1-38654705872 2-55834575048' 2026-02-17T12:22:46.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:46.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836695 2026-02-17T12:22:46.833 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:46.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:46.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836695 2026-02-17T12:22:46.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:46.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836695 2026-02-17T12:22:46.836 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836695' 2026-02-17T12:22:46.837 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836695 2026-02-17T12:22:46.837 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:46.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836693 -lt 21474836695 2026-02-17T12:22:46.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:47.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:47.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:48.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836696 -lt 21474836695 2026-02-17T12:22:48.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:48.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705872 2026-02-17T12:22:48.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:48.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:48.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705872 2026-02-17T12:22:48.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:48.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705872 2026-02-17T12:22:48.158 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705872 2026-02-17T12:22:48.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705872' 2026-02-17T12:22:48.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:48.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705872 -lt 38654705872 2026-02-17T12:22:48.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:48.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575048 2026-02-17T12:22:48.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:48.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:48.319 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575048 2026-02-17T12:22:48.319 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:48.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575048 2026-02-17T12:22:48.321 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575048 2026-02-17T12:22:48.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575048' 2026-02-17T12:22:48.321 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:48.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575048 -lt 55834575048 2026-02-17T12:22:48.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:48.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:48.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:48.624 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:48.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:48.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:48.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:48.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:48.634 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:48.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:48.782 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:48.782 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:48.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:48.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:48.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:48.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836698 2026-02-17T12:22:48.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836698 2026-02-17T12:22:48.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836698' 2026-02-17T12:22:48.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:48.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:48.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705875 2026-02-17T12:22:48.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705875 2026-02-17T12:22:48.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836698 1-38654705875' 2026-02-17T12:22:48.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:48.925 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:48.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575051 2026-02-17T12:22:48.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575051 2026-02-17T12:22:48.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836698 1-38654705875 2-55834575051' 2026-02-17T12:22:48.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:48.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836698 2026-02-17T12:22:48.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:48.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:48.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836698 2026-02-17T12:22:49.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:49.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836698 2026-02-17T12:22:49.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836698' 2026-02-17T12:22:49.002 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836698 2026-02-17T12:22:49.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:49.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836696 -lt 21474836698 2026-02-17T12:22:49.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:50.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:50.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:50.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836699 -lt 21474836698 2026-02-17T12:22:50.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:50.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705875 2026-02-17T12:22:50.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:50.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:50.311 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705875 2026-02-17T12:22:50.311 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:50.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705875 2026-02-17T12:22:50.312 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705875 2026-02-17T12:22:50.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705875' 2026-02-17T12:22:50.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:50.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705875 -lt 38654705875 2026-02-17T12:22:50.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:50.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575051 2026-02-17T12:22:50.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:50.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:50.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575051 2026-02-17T12:22:50.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575051 2026-02-17T12:22:50.481 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575051 2026-02-17T12:22:50.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575051' 2026-02-17T12:22:50.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:50.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575051 -lt 55834575051 2026-02-17T12:22:50.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:50.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:50.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:50.777 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:50.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:50.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:50.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:50.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:50.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:50.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:50.939 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:50.940 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:50.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:50.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:50.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:51.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836701 2026-02-17T12:22:51.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836701 2026-02-17T12:22:51.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836701' 2026-02-17T12:22:51.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:51.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:51.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705878 2026-02-17T12:22:51.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705878 2026-02-17T12:22:51.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836701 1-38654705878' 2026-02-17T12:22:51.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:51.082 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:51.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575054 2026-02-17T12:22:51.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575054 2026-02-17T12:22:51.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836701 1-38654705878 2-55834575054' 2026-02-17T12:22:51.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:51.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836701 2026-02-17T12:22:51.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:51.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:51.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836701 2026-02-17T12:22:51.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:51.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836701 2026-02-17T12:22:51.157 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836701 2026-02-17T12:22:51.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836701' 2026-02-17T12:22:51.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:51.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836699 -lt 21474836701 2026-02-17T12:22:51.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:52.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:52.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:52.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836702 -lt 21474836701 2026-02-17T12:22:52.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:52.474 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705878 2026-02-17T12:22:52.474 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:52.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:52.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705878 2026-02-17T12:22:52.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:52.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705878 2026-02-17T12:22:52.478 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705878 2026-02-17T12:22:52.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705878' 2026-02-17T12:22:52.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:52.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705878 -lt 38654705878 2026-02-17T12:22:52.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:52.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575054 2026-02-17T12:22:52.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:52.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:52.635 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575054 2026-02-17T12:22:52.635 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:52.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575054 2026-02-17T12:22:52.637 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575054 2026-02-17T12:22:52.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575054' 2026-02-17T12:22:52.637 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:52.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575054 -lt 55834575054 2026-02-17T12:22:52.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:52.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:52.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:52.942 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:52.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:52.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:52.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:52.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:52.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:53.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:53.102 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:53.102 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:53.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:53.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:53.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:53.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836705 2026-02-17T12:22:53.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836705 2026-02-17T12:22:53.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836705' 2026-02-17T12:22:53.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:53.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:53.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705881 2026-02-17T12:22:53.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705881 2026-02-17T12:22:53.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836705 1-38654705881' 2026-02-17T12:22:53.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:53.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:53.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575057 2026-02-17T12:22:53.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575057 2026-02-17T12:22:53.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836705 1-38654705881 2-55834575057' 2026-02-17T12:22:53.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:53.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836705 2026-02-17T12:22:53.321 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:53.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:53.322 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836705 2026-02-17T12:22:53.322 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:53.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836705 2026-02-17T12:22:53.324 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836705 2026-02-17T12:22:53.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836705' 2026-02-17T12:22:53.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:53.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836702 -lt 21474836705 2026-02-17T12:22:53.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:22:54.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:22:54.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:54.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836705 -lt 21474836705 2026-02-17T12:22:54.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:54.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705881 2026-02-17T12:22:54.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:54.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:54.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705881 2026-02-17T12:22:54.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:54.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705881 2026-02-17T12:22:54.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705881' 2026-02-17T12:22:54.632 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705881 2026-02-17T12:22:54.632 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:54.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705881 -lt 38654705881 2026-02-17T12:22:54.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:54.784 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575057 2026-02-17T12:22:54.784 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:54.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:54.786 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575057 2026-02-17T12:22:54.786 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:54.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575057 2026-02-17T12:22:54.788 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575057 2026-02-17T12:22:54.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575057' 2026-02-17T12:22:54.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:54.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575057 -lt 55834575057 2026-02-17T12:22:54.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:54.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:54.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:55.084 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:55.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:22:55.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:22:55.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:22:55.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:22:55.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:22:55.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:22:55.242 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:22:55.242 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:22:55.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:22:55.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:55.242 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:22:55.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836708 2026-02-17T12:22:55.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836708 2026-02-17T12:22:55.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836708' 2026-02-17T12:22:55.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:55.315 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:22:55.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705884 2026-02-17T12:22:55.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705884 2026-02-17T12:22:55.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836708 1-38654705884' 2026-02-17T12:22:55.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:22:55.385 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:22:55.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834575060 2026-02-17T12:22:55.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834575060 2026-02-17T12:22:55.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836708 1-38654705884 2-55834575060' 2026-02-17T12:22:55.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:55.458 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836708 2026-02-17T12:22:55.458 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:55.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:22:55.460 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836708 2026-02-17T12:22:55.460 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:55.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836708 2026-02-17T12:22:55.461 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836708 2026-02-17T12:22:55.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836708' 2026-02-17T12:22:55.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:22:55.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836708 -lt 21474836708 2026-02-17T12:22:55.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:55.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705884 2026-02-17T12:22:55.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:55.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:22:55.622 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705884 2026-02-17T12:22:55.622 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:55.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705884 2026-02-17T12:22:55.624 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705884 2026-02-17T12:22:55.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705884' 2026-02-17T12:22:55.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:22:55.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705884 -lt 38654705884 2026-02-17T12:22:55.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:22:55.784 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834575060 2026-02-17T12:22:55.784 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:22:55.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:22:55.786 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834575060 2026-02-17T12:22:55.786 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:22:55.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834575060 2026-02-17T12:22:55.788 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834575060 2026-02-17T12:22:55.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834575060' 2026-02-17T12:22:55.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:22:55.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834575060 -lt 55834575060 2026-02-17T12:22:55.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:22:55.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:22:55.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:22:56.084 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:22:56.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:383: _scrub_abort: break 2026-02-17T12:22:56.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:385: _scrub_abort: set +o pipefail 2026-02-17T12:22:56.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:387: _scrub_abort: sleep 5 2026-02-17T12:23:01.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:389: _scrub_abort: grep 'nodeep_scrub set, aborting' td/osd-scrub-test/osd.1.log 2026-02-17T12:23:01.103 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T12:20:58.415+0000 7f67cd31e640 10 osd.1 pg_epoch: 18 pg[1.0( v 16'1000 (0'0,16'1000] local-lis/les=14/15 n=1000 ec=14/14 lis/c=14/14 les/c/f=15/15/0 sis=14) [1,0,2] r=0 lpr=14 crt=16'1000 lcod 16'999 mlcod 16'999 active+clean+scrubbing+deep [ 1.0: ] TIME_FOR_DEEP] scrubber: nodeep_scrub set, aborting 2026-02-17T12:23:01.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:395: _scrub_abort: get_last_scrub_stamp 1.0 2026-02-17T12:23:01.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:01.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:01.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:01.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:01.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:395: _scrub_abort: local last_scrub=2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:01.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:396: _scrub_abort: ceph config set osd osd_scrub_sleep 0.1 2026-02-17T12:23:01.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:398: _scrub_abort: ceph osd unset nodeep-scrub 2026-02-17T12:23:01.603 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is unset 2026-02-17T12:23:01.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:399: _scrub_abort: '[' deep-scrub = deep-scrub ']' 2026-02-17T12:23:01.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:401: _scrub_abort: ceph osd unset noscrub 2026-02-17T12:23:01.803 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is unset 2026-02-17T12:23:01.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:403: _scrub_abort: TIMEOUT=500 2026-02-17T12:23:01.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:404: _scrub_abort: wait_for_scrub 1.0 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:01.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T12:23:01.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:01.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:23:01.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:23:01.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:01.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:01.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:01.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:01.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:01.817 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:01.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:01.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:02.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:02.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:02.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:02.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:02.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:02.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:02.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:03.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:03.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:04.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:04.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:04.120 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:04.120 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:04.120 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:04.120 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:04.120 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:04.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:04.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:05.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:05.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:05.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:05.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:05.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:05.269 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:05.269 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:05.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:05.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:06.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:06.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:06.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:06.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:06.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:06.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:06.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:06.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:06.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:07.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:07.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:07.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:07.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:07.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:07.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:07.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:07.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:07.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:08.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:08.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:08.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:08.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:08.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:08.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:08.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:08.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:08.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:09.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:09.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:09.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:09.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:09.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:09.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:09.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:10.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:10.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:11.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:11.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:11.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:11.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:11.018 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:11.018 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:11.018 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:11.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:11.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:12.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:12.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:12.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:12.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:12.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:12.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:12.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:12.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:12.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:13.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:13.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:13.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:13.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:13.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:13.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:13.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:13.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:13.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:14.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:14.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:14.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:14.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:14.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:14.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:14.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:14.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:14.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:15.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:15.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:15.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:15.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:15.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:15.617 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:15.617 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:15.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:15.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:16.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:16.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:16.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:16.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:16.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:16.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:16.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:16.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:16.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:17.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:17.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:17.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:17.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:17.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:17.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:17.928 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:18.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:18.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:19.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:19.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:19.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:19.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:19.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:19.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:19.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:19.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:19.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:20.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:20.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:20.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:20.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:20.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:20.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:20.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:20.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:20.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:21.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:21.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:21.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:21.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:21.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:21.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:21.396 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:21.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:21.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:22.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:22.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:22.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:22.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:22.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:22.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:22.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:22.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:22.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:23.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:23.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:23.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:23.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:23.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:23.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:23.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:23.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:23.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:24.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:24.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:24.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:24.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:24.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:24.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:24.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:24.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:24.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:25.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:25.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:25.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:25.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:25.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:25.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:25.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:26.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:26.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:27.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:27.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:27.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:27.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:27.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:27.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:27.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:27.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:27.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:28.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:28.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:28.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:28.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:28.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:28.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:28.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:28.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:28.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:29.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:29.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:29.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:29.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:29.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:29.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:29.462 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:29.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:29.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:30.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:30.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:30.617 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:30.617 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:30.617 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:30.617 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:30.617 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:30.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:30.769 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:31.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:31.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:31.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:31.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:31.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:31.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:31.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:31.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:31.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:32.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:32.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:32.920 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:32.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:32.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:32.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:32.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:33.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:33.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:34.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:34.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:34.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:34.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:34.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:34.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:34.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:34.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:34.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:35.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:35.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:35.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:35.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:35.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:35.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:35.222 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:35.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:35.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:36.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:36.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:36.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:36.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:36.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:36.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:36.371 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:36.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:36.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:37.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:37.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:37.523 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:37.523 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:37.524 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:37.524 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:37.524 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:37.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:37.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:38.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:38.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:38.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:38.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:38.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:38.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:38.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:38.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:38.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:39.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:39.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:39.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:39.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:39.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:39.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:39.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:39.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:39.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:40.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:40.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:40.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:40.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:40.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:40.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:40.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:41.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:41.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:42.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:42.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:42.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:42.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:42.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:42.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:42.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:42.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:42.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:43.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:43.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:43.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:43.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:43.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:43.278 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:43.278 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:43.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:43.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:44.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:44.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:44.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:44.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:44.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:44.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:44.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:44.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:44.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:45.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:45.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:45.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:45.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:45.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:45.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:45.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:45.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:45.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:46.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:46.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:46.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:46.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:46.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:46.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:46.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:46.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:46.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:47.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:47.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:47.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:47.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:47.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:47.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:47.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:48.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:48.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:49.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:49.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:49.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:49.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:49.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:49.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:49.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:49.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:49.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:50.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:50.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:50.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:50.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:50.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:50.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:50.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:50.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:50.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:51.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:51.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:51.342 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:51.342 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:51.342 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:51.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:51.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:51.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:51.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:52.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:52.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:52.497 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:52.498 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:52.498 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:52.499 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:52.499 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:52.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:52.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:53.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:53.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:53.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:53.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:53.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:53.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:53.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:53.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:53.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:54.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:54.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:54.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:54.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:54.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:54.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:54.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:54.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:54.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:55.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:55.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:55.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:55.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:55.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:55.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:55.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:56.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:56.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:57.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:57.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:57.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:57.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:57.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:57.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:57.127 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:57.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:57.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:58.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:58.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:58.279 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:58.279 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:58.280 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:58.280 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:58.280 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:58.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:58.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:23:59.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:23:59.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:23:59.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:23:59.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:23:59.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:23:59.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:23:59.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:23:59.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:23:59.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:00.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:00.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:00.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:00.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:00.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:00.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:00.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:00.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:00.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:01.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:01.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:01.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:01.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:01.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:01.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:01.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:01.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:01.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:02.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:02.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:02.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:02.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:02.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:02.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:02.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:03.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:03.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:04.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:04.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:04.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:04.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:04.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:04.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:04.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:04.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:04.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:05.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:05.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:05.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:05.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:05.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:05.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:05.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:05.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:05.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:06.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:06.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:06.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:06.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:06.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:06.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:06.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:06.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:06.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:07.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:07.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:07.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:07.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:07.480 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:07.480 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:07.480 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:07.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:07.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:08.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:08.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:08.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:08.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:08.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:08.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:08.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:08.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:08.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:09.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:09.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:09.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:09.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:09.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:09.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:09.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:09.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:09.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:10.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:10.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:10.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:10.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:10.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:10.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:10.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:11.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:11.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:12.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:12.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:12.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:12.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:12.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:12.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:12.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:12.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:12.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:13.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:13.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:13.250 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:13.250 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:13.250 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:13.251 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:13.251 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:13.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:13.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:14.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:14.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:14.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:14.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:14.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:14.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:14.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:14.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:14.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:15.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:15.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:15.552 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:15.552 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:15.553 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:15.553 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:15.553 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:15.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:15.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:16.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:16.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:16.703 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:16.703 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:16.703 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:16.703 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:16.703 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:16.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:16.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:17.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:17.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:17.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:17.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:17.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:17.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:17.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:18.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:18.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:19.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:19.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:19.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:19.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:19.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:19.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:19.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:19.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:19.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:20.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:20.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:20.156 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:20.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:20.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:20.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:20.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:20.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:20.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:21.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:21.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:21.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:21.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:21.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:21.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:21.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:21.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:21.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:22.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:22.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:22.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:22.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:22.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:22.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:22.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:22.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:22.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:23.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:23.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:23.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:23.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:23.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:23.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:23.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:23.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:23.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:24.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:24.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:24.757 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:24.757 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:24.757 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:24.757 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:24.757 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:24.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:24.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:25.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:25.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:25.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:25.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:25.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:25.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:25.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:26.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:26.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:27.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:27.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:27.057 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:27.057 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:27.057 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:27.057 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:27.057 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:27.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:27.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:28.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:28.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:28.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:28.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:28.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:28.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:28.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:28.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:28.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:29.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:29.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:29.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:29.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:29.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:29.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:29.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:29.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:29.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:30.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:30.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:30.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:30.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:30.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:30.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:30.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:30.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:30.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:31.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:31.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:31.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:31.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:31.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:31.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:31.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:31.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:31.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:32.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:32.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:32.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:32.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:32.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:32.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:32.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:32.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:32.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:33.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:33.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:33.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:33.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:33.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:33.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:33.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:34.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:34.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:35.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:35.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:35.139 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:35.139 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:35.139 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:35.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:35.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:35.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:35.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:36.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:36.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:36.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:36.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:36.294 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:36.294 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:36.294 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:36.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:36.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:37.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:37.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:37.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:37.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:37.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:37.448 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:37.448 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:37.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:37.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:38.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:38.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:38.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:38.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:38.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:38.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:38.603 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:38.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:38.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:39.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:39.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:39.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:39.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:39.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:39.759 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:39.759 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:39.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:39.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:40.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:40.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:40.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:40.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:40.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:40.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:40.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:41.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:41.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:42.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:42.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:42.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:42.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:42.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:42.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:42.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:42.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:42.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:43.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:43.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:43.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:43.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:43.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:43.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:43.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:43.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:43.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:44.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:44.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:44.379 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:44.380 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:44.380 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:44.380 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:44.380 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:44.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:44.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:45.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:45.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:45.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:45.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:45.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:45.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:45.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:45.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:45.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:46.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:46.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:46.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:46.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:46.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:46.686 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:46.686 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:46.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:46.839 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:47.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:47.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:47.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:47.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:47.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:47.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:47.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:47.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:47.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:48.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:48.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:48.992 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:48.992 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:48.992 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:48.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:48.993 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:49.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:49.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:50.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:50.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:50.146 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:50.146 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:50.146 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:50.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:50.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:50.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:50.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:51.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:51.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:51.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:51.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:51.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:51.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:51.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:51.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:51.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:52.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:52.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:52.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:52.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:52.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:52.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:52.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:52.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:52.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:53.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:53.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:53.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:53.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:53.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:53.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:53.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:53.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:53.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:54.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:54.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:54.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:54.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:54.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:54.749 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:54.749 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:54.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:54.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:55.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:55.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:55.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:55.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:55.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:55.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:55.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:56.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:56.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:57.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:57.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:57.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:57.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:57.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:57.054 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:57.054 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:57.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:57.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:58.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:58.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:58.205 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:58.205 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:58.205 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:58.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:58.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:58.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:58.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:24:59.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:24:59.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:24:59.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:24:59.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:24:59.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:24:59.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:24:59.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:24:59.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:24:59.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:00.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:00.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:00.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:00.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:00.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:00.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:00.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:00.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:00.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:01.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:01.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:01.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:01.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:01.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:01.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:01.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:01.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:01.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:02.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:02.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:02.807 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:02.807 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:02.807 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:02.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:02.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:02.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:02.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:03.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:03.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:03.959 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:03.959 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:03.959 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:03.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:03.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:04.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:04.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:05.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:05.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:05.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:05.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:05.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:05.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:05.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:05.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:05.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:06.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:06.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:06.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:06.260 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:06.260 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:06.260 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:06.260 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:06.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:06.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:07.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:07.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:07.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:07.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:07.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:07.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:07.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:07.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:07.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:08.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:08.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:08.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:08.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:08.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:08.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:08.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:08.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:08.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:09.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:09.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:09.705 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:09.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:09.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:09.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:09.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:09.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:09.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:10.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:10.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:10.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:10.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:10.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:10.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:10.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:10.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:10.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:12.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:12.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:12.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:12.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:12.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:12.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:12.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:12.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:12.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:13.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:13.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:13.151 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:13.151 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:13.151 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:13.151 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:13.151 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:13.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:13.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:14.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:14.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:14.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:14.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:14.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:14.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:14.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:14.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:14.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:15.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:15.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:15.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:15.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:15.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:15.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:15.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:15.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:15.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:16.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:16.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:16.610 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:16.610 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:16.610 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:16.611 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:16.611 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:16.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:16.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:17.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:17.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:17.760 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:17.760 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:17.760 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:17.761 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:17.761 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:17.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:17.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:18.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:18.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:18.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:18.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:18.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:18.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:18.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:19.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:19.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:20.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:20.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:20.074 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:20.074 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:20.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:20.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:20.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:20.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:20.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:21.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:21.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:21.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:21.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:21.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:21.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:21.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:21.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:21.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:22.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:22.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:22.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:22.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:22.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:22.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:22.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:22.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:22.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:23.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:23.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:23.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:23.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:23.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:23.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:23.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:23.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:23.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:24.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:24.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:24.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:24.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:24.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:24.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:24.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:24.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:24.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:25.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:25.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:25.850 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:25.850 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:25.850 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:25.850 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:25.850 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:26.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:26.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:27.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:27.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:27.003 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:27.003 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:27.003 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:27.004 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:27.004 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:27.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:27.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:28.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:28.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:28.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:28.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:28.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:28.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:28.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:28.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:28.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:29.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:29.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:29.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:29.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:29.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:29.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:29.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:29.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:29.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:30.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:30.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:30.464 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:30.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:30.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:30.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:30.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:30.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:30.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:31.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:31.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:31.614 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:31.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:31.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:31.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:31.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:31.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:31.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:32.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:32.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:32.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:32.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:32.764 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:32.764 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:32.764 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:32.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:32.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:33.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:33.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:33.913 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:33.913 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:33.913 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:33.914 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:33.914 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:34.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:34.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:35.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:35.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:35.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:35.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:35.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:35.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:35.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:35.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:35.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:36.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:36.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:36.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:36.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:36.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:36.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:36.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:36.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:36.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:37.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:37.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:37.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:37.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:37.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:37.361 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:37.361 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:37.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:37.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:38.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:38.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:38.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:38.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:38.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:38.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:38.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:38.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:38.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:39.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:39.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:39.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:39.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:39.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:39.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:39.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:39.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:39.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:40.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:40.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:40.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:40.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:40.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:40.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:40.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:40.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:40.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:41.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:41.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:41.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:41.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:41.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:41.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:41.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:42.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:42.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:43.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:43.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:43.139 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:43.139 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:43.139 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:43.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:43.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:43.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:43.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:44.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:44.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:44.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:44.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:44.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:44.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:44.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:44.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:44.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:45.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:45.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:45.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:45.452 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:45.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:45.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:45.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:45.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:45.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:46.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:46.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:46.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:46.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:46.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:46.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:46.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:46.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:46.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:47.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:47.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:47.767 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:47.767 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:47.767 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:47.767 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:47.768 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:47.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:47.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:48.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:48.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:48.924 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:48.924 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:48.924 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:48.925 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:48.925 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:49.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:49.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:50.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:50.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:50.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:50.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:50.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:50.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:50.077 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:50.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:50.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:51.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:51.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:51.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:51.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:51.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:51.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:51.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:51.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:51.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:52.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:52.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:52.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:52.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:52.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:52.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:52.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:52.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:52.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:53.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:53.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:53.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:53.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:53.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:53.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:53.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:53.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:53.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:54.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:54.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:54.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:54.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:54.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:54.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:54.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:54.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:54.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:55.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:55.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:55.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:55.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:55.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:55.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:55.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:56.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:56.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:57.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:57.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:57.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:57.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:57.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:57.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:57.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:57.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:57.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:58.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:58.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:58.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:58.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:58.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:58.162 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:58.162 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:58.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:58.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:25:59.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:25:59.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:25:59.314 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:25:59.314 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:25:59.315 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:25:59.315 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:25:59.315 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:25:59.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:25:59.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:00.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:00.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:00.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:00.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:00.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:00.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:00.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:00.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:00.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:01.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:01.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:01.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:01.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:01.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:01.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:01.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:01.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:01.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:02.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:02.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:02.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:02.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:02.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:02.764 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:02.764 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:02.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:02.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:03.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:03.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:03.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:03.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:03.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:03.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:03.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:04.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:04.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:05.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:05.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:05.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:05.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:05.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:05.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:05.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:05.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:05.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:06.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:06.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:06.218 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:06.219 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:06.219 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:06.219 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:06.219 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:06.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:06.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:07.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:07.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:07.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:07.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:07.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:07.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:07.367 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:07.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:07.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:08.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:08.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:08.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:08.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:08.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:08.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:08.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:08.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:08.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:09.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:09.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:09.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:09.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:09.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:09.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:09.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:09.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:09.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:10.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:10.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:10.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:10.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:10.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:10.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:10.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:10.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:10.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:11.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:11.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:11.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:11.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:11.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:11.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:11.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:12.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:12.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:13.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:13.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:13.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:13.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:13.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:13.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:13.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:13.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:13.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:14.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:14.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:14.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:14.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:14.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:14.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:14.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:14.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:14.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:15.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:15.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:15.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:15.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:15.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:15.421 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:15.421 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:15.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:15.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:16.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:16.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:16.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:16.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:16.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:16.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:16.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:16.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:16.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:17.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:17.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:17.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:17.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:17.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:17.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:17.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:17.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:17.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:18.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:18.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:18.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:18.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:18.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:18.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:18.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:19.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:19.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:20.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:20.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:20.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:20.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:20.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:20.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:20.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:20.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:20.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:21.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:21.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:21.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:21.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:21.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:21.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:21.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:21.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:21.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:22.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:22.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:22.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:22.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:22.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:22.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:22.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:22.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:22.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:23.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:23.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:23.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:23.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:23.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:23.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:23.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:23.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:23.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:24.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:24.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:24.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:24.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:24.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:24.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:24.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:24.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:24.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:25.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:25.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:25.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:25.778 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:25.778 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:25.778 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:25.778 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:25.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:25.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:26.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:26.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:26.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:26.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:26.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:26.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:26.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:27.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:27.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:28.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:28.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:28.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:28.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:28.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:28.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:28.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:28.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:28.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:29.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:29.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:29.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:29.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:29.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:29.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:29.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:29.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:29.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:30.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:30.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:30.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:30.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:30.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:30.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:30.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:30.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:30.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:31.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:31.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:31.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:31.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:31.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:31.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:31.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:31.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:31.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:32.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:32.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:32.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:32.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:32.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:32.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:32.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:32.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:32.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:33.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:33.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:33.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:33.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:33.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:33.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:33.847 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:33.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:33.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:34.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:34.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:34.999 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:35.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:35.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:35.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:35.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:35.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:35.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:36.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:36.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:36.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:36.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:36.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:36.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:36.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:36.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:36.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:37.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:37.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:37.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:37.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:37.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:37.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:37.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:37.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:37.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:38.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:38.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:38.464 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:38.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:38.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:38.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:38.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:38.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:38.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:39.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:39.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:39.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:39.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:39.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:39.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:39.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:39.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:39.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:40.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:40.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:40.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:40.766 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:40.766 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:40.766 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:40.766 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:40.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:40.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:41.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:41.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:41.914 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:41.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:41.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:41.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:41.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:42.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:42.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:43.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:43.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:43.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:43.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:43.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:43.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:43.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:43.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:43.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:44.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:44.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:44.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:44.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:44.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:44.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:44.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:44.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:44.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:45.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:45.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:45.379 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:45.379 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:45.379 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:45.379 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:45.379 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:45.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:45.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:46.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:46.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:46.536 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:46.536 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:46.536 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:46.536 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:46.536 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:46.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:46.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:47.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:47.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:47.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:47.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:47.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:47.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:47.692 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:47.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:47.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:48.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:48.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:48.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:48.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:48.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:48.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:48.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:48.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:48.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:49.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:49.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:49.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:49.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:49.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:49.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:49.995 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:50.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:50.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:51.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:51.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:51.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:51.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:51.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:51.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:51.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:51.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:51.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:52.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:52.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:52.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:52.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:52.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:52.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:52.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:52.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:52.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:53.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:53.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:53.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:53.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:53.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:53.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:53.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:53.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:53.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:54.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:54.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:54.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:54.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:54.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:54.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:54.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:54.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:54.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:55.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:55.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:55.761 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:55.761 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:55.761 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:55.761 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:55.761 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:55.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:55.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:56.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:56.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:56.909 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:56.909 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:56.910 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:56.910 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:56.910 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:57.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:57.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:58.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:58.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:58.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:58.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:58.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:58.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:58.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:58.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:58.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:26:59.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:26:59.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:26:59.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:26:59.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:26:59.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:26:59.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:26:59.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:26:59.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:26:59.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:00.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:00.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:00.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:00.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:00.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:00.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:00.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:00.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:00.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:01.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:01.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:01.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:01.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:01.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:01.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:01.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:01.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:01.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:02.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:02.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:02.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:02.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:02.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:02.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:02.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:02.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:02.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:03.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:03.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:03.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:03.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:03.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:03.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:03.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:03.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:03.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:04.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:04.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:04.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:04.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:04.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:04.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:04.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:05.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:05.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:06.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:06.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:06.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:06.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:06.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:06.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:06.144 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:06.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:06.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:07.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:07.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:07.299 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:07.299 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:07.299 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:07.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:07.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:07.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:07.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:08.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:08.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:08.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:08.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:08.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:08.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:08.454 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:08.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:08.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:09.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:09.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:09.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:09.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:09.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:09.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:09.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:09.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:09.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:10.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:10.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:10.762 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:10.762 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:10.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:10.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:10.763 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:10.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:10.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:11.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:11.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:11.913 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:11.914 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:11.914 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:11.914 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:11.914 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:12.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:12.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:13.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:13.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:13.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:13.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:13.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:13.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:13.063 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:13.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:13.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:14.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:14.211 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:14.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:14.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:14.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:14.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:14.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:14.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:14.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:15.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:15.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:15.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:15.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:15.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:15.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:15.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:15.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:15.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:16.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:16.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:16.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:16.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:16.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:16.507 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:16.507 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:16.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:16.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:17.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:17.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:17.655 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:17.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:17.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:17.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:17.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:17.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:17.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:18.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:18.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:18.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:18.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:18.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:18.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:18.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:18.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:18.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:19.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:19.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:19.962 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:19.962 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:19.962 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:19.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:19.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:20.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:20.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:21.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:21.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:21.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:21.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:21.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:21.118 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:21.118 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:21.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:21.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:22.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:22.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:22.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:22.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:22.272 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:22.272 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:22.272 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:22.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:22.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:23.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:23.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:23.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:23.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:23.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:23.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:23.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:23.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:23.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:24.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:24.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:24.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:24.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:24.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:24.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:24.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:24.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:24.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:25.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:25.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:25.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:25.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:25.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:25.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:25.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:25.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:25.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:26.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:26.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:26.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:26.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:26.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:26.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:26.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:27.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:27.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:28.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:28.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:28.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:28.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:28.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:28.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:28.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:28.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:28.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:29.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:29.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:29.204 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:29.204 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:29.204 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:29.204 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:29.205 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:29.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:29.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:30.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:30.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:30.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:30.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:30.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:30.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:30.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:30.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:30.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:31.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:31.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:31.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:31.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:31.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:31.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:31.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:31.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:31.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:32.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:32.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:32.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:32.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:32.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:32.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:32.670 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:32.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:32.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:33.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:33.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:33.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:33.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:33.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:33.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:33.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:33.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:33.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:34.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:34.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:34.974 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:34.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:34.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:34.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:34.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:35.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:35.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:36.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:36.128 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:36.130 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:36.130 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:36.130 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:36.130 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:36.130 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:36.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:36.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:37.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:37.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:37.285 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:37.285 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:37.285 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:37.286 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:37.286 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:37.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:37.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:38.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:38.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:38.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:38.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:38.439 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:38.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:38.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:38.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:38.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:39.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:39.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:39.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:39.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:39.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:39.596 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:39.596 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:39.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:39.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:40.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:40.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:40.745 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:40.745 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:40.745 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:40.745 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:40.745 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:40.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:40.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:41.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:41.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:41.895 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:41.895 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:41.895 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:41.895 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:41.895 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:42.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:42.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:43.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:43.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:43.044 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:43.044 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:43.044 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:43.044 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:43.044 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:43.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:43.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:44.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:44.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:44.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:44.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:44.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:44.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:44.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:44.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:44.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:45.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:45.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:45.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:45.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:45.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:45.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:45.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:45.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:45.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:46.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:46.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:46.493 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:46.493 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:46.493 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:46.493 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:46.494 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:46.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:46.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:47.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:47.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:47.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:47.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:47.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:47.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:47.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:47.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:47.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:48.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:48.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:48.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:48.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:48.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:48.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:48.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:48.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:48.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:49.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:49.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:49.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:49.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:49.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:49.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:49.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:50.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:50.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:51.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:51.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:51.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:51.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:51.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:51.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:51.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:51.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:51.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:52.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:52.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:52.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:52.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:52.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:52.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:52.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:52.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:52.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:53.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:53.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:53.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:53.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:53.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:53.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:53.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:53.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:53.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:54.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:54.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:54.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:54.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:54.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:54.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:54.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:54.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:54.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:55.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:55.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:55.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:55.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:55.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:55.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:55.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:55.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:55.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:56.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:56.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:56.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:56.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:56.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:56.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:56.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:57.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:57.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:58.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:58.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:58.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:58.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:58.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:58.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:58.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:58.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:58.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:27:59.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:27:59.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:27:59.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:27:59.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:27:59.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:27:59.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:27:59.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:27:59.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:27:59.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:00.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:00.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:00.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:00.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:00.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:00.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:00.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:00.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:00.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:01.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:01.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:01.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:01.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:01.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:01.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:01.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:01.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:01.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:02.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:02.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:02.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:02.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:02.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:02.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:02.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:03.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:03.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:03.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:03.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:03.798 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:03.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:03.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:03.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:03.799 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:03.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:03.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:04.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:04.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:04.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:04.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:04.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:04.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:04.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:05.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:05.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:06.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:06.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:06.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:06.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:06.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:06.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:06.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:06.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:06.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:07.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:07.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:07.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:07.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:07.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:07.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:07.245 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:07.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:07.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:08.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:08.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:08.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:08.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:08.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:08.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:08.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:08.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:08.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:09.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:09.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:09.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:09.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:09.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:09.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:09.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:09.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:09.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:10.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:10.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:10.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:10.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:10.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:10.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:10.695 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:10.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:10.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:11.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:11.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:11.850 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:11.850 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:11.850 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:11.851 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:11.851 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:12.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:12.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:13.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:13.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:13.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:13.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:13.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:13.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:13.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:13.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:13.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:14.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:14.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:14.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:14.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:14.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:14.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:14.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:14.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:14.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:15.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:15.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:15.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:15.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:15.314 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:15.314 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:15.314 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:15.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:15.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:16.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:16.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:16.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:16.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:16.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:16.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:16.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:16.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:16.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:17.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:17.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:17.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:17.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:17.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:17.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:17.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:17.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:17.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:18.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:18.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:18.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:18.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:18.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:18.774 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:18.774 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:18.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:18.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:19.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:19.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:19.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:19.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:19.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:19.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:19.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:20.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:20.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:21.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:21.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:21.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:21.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:21.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:21.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:21.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:21.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:21.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:22.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:22.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:22.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:22.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:22.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:22.241 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:22.241 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:22.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:22.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:23.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:23.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:23.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:23.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:23.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:23.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:23.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:23.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:23.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:24.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:24.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:24.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:24.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:24.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:24.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:24.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:24.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:24.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:25.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:25.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:25.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:25.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:25.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:25.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:25.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:25.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:25.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:26.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:26.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:26.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:26.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:26.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:26.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:26.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:26.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:26.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:27.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:27.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:27.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:27.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:27.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:27.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:27.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:28.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:28.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:29.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:29.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:29.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:29.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:29.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:29.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:29.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:29.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:29.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:30.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:30.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:30.299 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:30.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:30.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:30.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:30.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:30.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:30.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:31.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:31.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:31.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:31.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:31.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:31.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:31.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:31.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:31.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:32.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:32.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:32.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:32.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:32.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:32.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:32.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:32.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:32.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:33.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:33.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:33.753 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:33.753 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:33.753 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:33.753 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:33.753 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:33.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:33.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:34.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:34.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:34.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:34.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:34.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:34.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:34.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:35.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:35.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:36.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:36.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:36.056 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:36.056 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:36.056 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:36.056 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:36.056 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:36.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:36.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:37.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:37.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:37.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:37.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:37.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:37.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:37.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:37.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:37.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:38.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:38.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:38.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:38.355 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:38.355 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:38.355 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:38.355 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:38.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:38.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:39.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:39.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:39.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:39.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:39.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:39.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:39.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:39.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:39.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:40.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:40.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:40.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:40.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:40.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:40.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:40.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:40.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:40.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:41.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:41.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:41.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:41.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:41.803 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:41.804 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:41.804 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:41.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:41.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:42.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:42.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:42.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:42.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:42.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:42.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:42.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:43.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-03T12:19:10.578762+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:43.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:28:44.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:28:44.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:28:44.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:28:44.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:28:44.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:28:44.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:28:44.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:28:44.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:28:42.283860+0000 '>' 2026-02-03T12:19:10.578762+0000 2026-02-17T12:28:44.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:28:44.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:405: _scrub_abort: perf_counters td/osd-scrub-test 3 2026-02-17T12:28:44.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-17T12:28:44.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-17T12:28:44.249 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-17T12:28:44.250 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-17T12:28:44.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:28:44.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-17T12:28:44.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:28:44.324 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.325 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:28:44.326 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:28:44.327 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:28:44.328 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.329 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:28:44.330 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.331 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:28:44.332 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:28:44.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:28:44.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-17T12:28:44.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:28:44.397 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:28:44.397 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:28:44.397 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:28:44.397 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:28:44.397 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:28:44.397 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:28:44.397 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.397 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:28:44.397 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:28:44.397 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:28:44.397 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:28:44.397 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 2, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 2, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 1, 2026-02-17T12:28:44.398 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 1, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 335.27908236, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 335.27908236 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 1, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 1, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 4.999920892, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 4.999920892 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 67, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 2, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 2, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 2 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:28:44.399 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.400 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.401 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.401 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:28:44.401 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:28:44.401 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.401 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.401 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.401 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.401 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:28:44.401 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:28:44.401 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:28:44.401 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:28:44.401 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:28:44.401 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:28:44.403 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:28:44.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:28:44.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-17T12:28:44.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:28:44.465 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:28:44.465 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:28:44.465 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:28:44.465 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:28:44.465 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:28:44.465 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:28:44.465 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.465 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:28:44.465 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:28:44.465 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:28:44.465 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:28:44.465 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:28:44.466 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.467 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.468 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.469 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:28:44.469 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:28:44.469 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.469 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.469 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.469 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.469 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:28:44.469 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:28:44.469 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:28:44.469 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:28:44.469 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:28:44.469 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:28:44.469 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:28:44.471 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.471 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.471 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Teardown Test TEST_deep_scrub_abort ------------------ 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_deep_scrub_abort ------------------' 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:28:44.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:28:44.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:28:44.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:28:44.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:28:44.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:28:44.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:28:44.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:28:44.582 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:28:44.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:28:44.583 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:28:44.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:28:44.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:28:44.584 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:28:44.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:28:44.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:28:44.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:28:44.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:28:44.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:28:44.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:28:44.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:28:44.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:28:44.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:28:44.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:28:44.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:28:44.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:28:44.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:28:44.651 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Complete Test TEST_deep_scrub_abort ------------------ 2026-02-17T12:28:44.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_deep_scrub_abort ------------------' 2026-02-17T12:28:44.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T12:28:44.652 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Prepare Test TEST_dump_scrub_schedule ------------------- 2026-02-17T12:28:44.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_dump_scrub_schedule -------------------' 2026-02-17T12:28:44.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T12:28:44.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T12:28:44.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T12:28:44.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:28:44.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:28:44.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:28:44.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:28:44.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:28:44.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:28:44.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:28:44.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:28:44.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:28:44.655 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:28:44.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:28:44.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:28:44.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:28:44.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:28:44.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:28:44.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:28:44.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:28:44.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:28:44.659 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:28:44.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:28:44.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:28:44.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:28:44.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:28:44.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:28:44.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:28:44.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:28:44.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:28:44.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:28:44.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T12:28:44.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:28:44.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:28:44.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:28:44.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.508089 2026-02-17T12:28:44.667 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:28:44.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:28:44.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:28:44.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T12:28:44.668 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Run Test TEST_dump_scrub_schedule ----------------------- 2026-02-17T12:28:44.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_dump_scrub_schedule -----------------------' 2026-02-17T12:28:44.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_dump_scrub_schedule td/osd-scrub-test 2026-02-17T12:28:44.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:530: TEST_dump_scrub_schedule: local dir=td/osd-scrub-test 2026-02-17T12:28:44.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:531: TEST_dump_scrub_schedule: local poolname=test 2026-02-17T12:28:44.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:532: TEST_dump_scrub_schedule: local OSDS=3 2026-02-17T12:28:44.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:533: TEST_dump_scrub_schedule: local objects=90 2026-02-17T12:28:44.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:535: TEST_dump_scrub_schedule: TESTDATA=testdata.508089 2026-02-17T12:28:44.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:537: TEST_dump_scrub_schedule: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-17T12:28:44.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-test 2026-02-17T12:28:44.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:28:44.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:28:44.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:28:44.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-test/a 2026-02-17T12:28:44.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-17T12:28:44.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:28:44.689 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:28:44.689 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:28:44.689 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:28:44.689 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:28:44.689 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:28:44.689 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:28:44.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-17T12:28:44.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:28:44.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:28:44.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:28:44.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:28:44.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:28:44.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:28:44.720 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:28:44.720 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:28:44.721 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:28:44.721 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:28:44.721 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:28:44.721 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:28:44.721 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-mon.a.asok 2026-02-17T12:28:44.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:28:44.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.508089/ceph-mon.a.asok config get fsid 2026-02-17T12:28:44.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:28:44.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:28:44.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:28:44.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:28:44.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:28:44.781 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:28:44.781 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:28:44.781 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:28:44.781 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:28:44.781 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:28:44.781 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:28:44.781 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-mon.a.asok 2026-02-17T12:28:44.781 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:28:44.781 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.508089/ceph-mon.a.asok config get mon_host 2026-02-17T12:28:44.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:538: TEST_dump_scrub_schedule: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-17T12:28:44.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-test 2026-02-17T12:28:44.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:28:44.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:28:44.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:28:44.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-test/x 2026-02-17T12:28:44.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:28:44.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:28:44.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:28:44.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:28:44.927 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:28:44.928 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:28:44.928 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:28:44.929 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:28:44.929 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:28:44.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T12:28:44.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:549: TEST_dump_scrub_schedule: local 'ceph_osd_args=--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2' 2026-02-17T12:28:44.947 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:551: TEST_dump_scrub_schedule: expr 3 - 1 2026-02-17T12:28:44.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:551: TEST_dump_scrub_schedule: seq 0 2 2026-02-17T12:28:44.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:551: TEST_dump_scrub_schedule: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:28:44.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:553: TEST_dump_scrub_schedule: run_osd td/osd-scrub-test 0 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 2026-02-17T12:28:44.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:28:44.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:28:44.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:28:44.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:28:44.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-17T12:28:44.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:28:44.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:28:44.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:28:44.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:28:44.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-17T12:28:44.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-17T12:28:44.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:28:44.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:28:44.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:28:44.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:28:44.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:28:44.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:28:44.953 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:28:44.954 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:28:44.954 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:28:44.954 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:28:44.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:28:44.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:28:44.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:28:44.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:28:44.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:28:44.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:28:44.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:28:44.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:28:44.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:28:44.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:28:44.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:28:44.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2' 2026-02-17T12:28:44.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-17T12:28:44.956 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:28:44.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=995348b7-617e-4f5e-b86d-f704ee48237e 2026-02-17T12:28:44.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 995348b7-617e-4f5e-b86d-f704ee48237e' 2026-02-17T12:28:44.956 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 995348b7-617e-4f5e-b86d-f704ee48237e 2026-02-17T12:28:44.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:28:44.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQD8XpRpHCUeORAAwQcmvaopNEALf6CmdyaylA== 2026-02-17T12:28:44.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQD8XpRpHCUeORAAwQcmvaopNEALf6CmdyaylA=="}' 2026-02-17T12:28:44.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 995348b7-617e-4f5e-b86d-f704ee48237e -i td/osd-scrub-test/0/new.json 2026-02-17T12:28:45.053 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:28:45.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-17T12:28:45.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 --mkfs --key AQD8XpRpHCUeORAAwQcmvaopNEALf6CmdyaylA== --osd-uuid 995348b7-617e-4f5e-b86d-f704ee48237e 2026-02-17T12:28:45.078 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:45.072+0000 7f459e7ec8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:28:45.079 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:45.072+0000 7f459e7ec8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:28:45.080 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:45.072+0000 7f459e7ec8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:28:45.081 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:45.072+0000 7f459e7ec8c0 -1 bdev(0x55ec095adc00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:28:45.081 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:45.072+0000 7f459e7ec8c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T12:28:47.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-17T12:28:47.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:28:47.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:28:47.896 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:28:47.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:28:48.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:28:48.087 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:28:48.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 2026-02-17T12:28:48.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:28:48.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:28:48.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:28:48.111 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:48.104+0000 7f4aae04e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:28:48.112 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:48.104+0000 7f4aae04e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:28:48.113 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:48.104+0000 7f4aae04e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:28:48.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:28:48.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:28:48.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:28:48.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:28:48.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:28:48.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:28:48.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:28:48.244 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:28:48.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:28:48.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:28:48.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:28:49.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:28:49.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:28:49.395 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:28:49.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:28:49.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:28:49.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:28:49.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:28:49.595 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:49.587+0000 7f4aae04e8c0 -1 Falling back to public interface 2026-02-17T12:28:50.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:28:50.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:28:50.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:28:50.546 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:28:50.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:28:50.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:28:50.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:28:51.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:28:51.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:28:51.695 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:28:51.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:28:51.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:28:51.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:28:51.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:28:52.651 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:52.643+0000 7f4aae04e8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:28:52.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:28:52.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:28:52.846 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:28:52.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:28:52.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:28:52.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:28:52.997 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/101294961,v1:127.0.0.1:6803/101294961] [v2:127.0.0.1:6804/101294961,v1:127.0.0.1:6805/101294961] exists,up 995348b7-617e-4f5e-b86d-f704ee48237e 2026-02-17T12:28:53.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:28:53.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:28:53.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:28:53.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:551: TEST_dump_scrub_schedule: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:28:53.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:553: TEST_dump_scrub_schedule: run_osd td/osd-scrub-test 1 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 2026-02-17T12:28:53.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:28:53.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:28:53.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:28:53.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:28:53.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-17T12:28:53.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:28:53.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:28:53.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:28:53.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:28:53.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-17T12:28:53.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-17T12:28:53.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:28:53.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:28:53.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:28:53.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:28:53.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:28:53.002 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:28:53.003 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:28:53.003 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:28:53.003 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:28:53.003 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:28:53.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:28:53.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:28:53.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:28:53.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:28:53.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:28:53.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:28:53.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:28:53.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:28:53.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:28:53.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:28:53.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:28:53.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2' 2026-02-17T12:28:53.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-17T12:28:53.004 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:28:53.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e1acd1cf-ff5f-4e6f-b90d-63968b6abed5 2026-02-17T12:28:53.005 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 e1acd1cf-ff5f-4e6f-b90d-63968b6abed5 2026-02-17T12:28:53.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 e1acd1cf-ff5f-4e6f-b90d-63968b6abed5' 2026-02-17T12:28:53.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:28:53.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAFX5RpM3OAABAA8v5lWg1/MALOGSid+3wKtw== 2026-02-17T12:28:53.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAFX5RpM3OAABAA8v5lWg1/MALOGSid+3wKtw=="}' 2026-02-17T12:28:53.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e1acd1cf-ff5f-4e6f-b90d-63968b6abed5 -i td/osd-scrub-test/1/new.json 2026-02-17T12:28:53.167 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:28:53.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-17T12:28:53.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 --mkfs --key AQAFX5RpM3OAABAA8v5lWg1/MALOGSid+3wKtw== --osd-uuid e1acd1cf-ff5f-4e6f-b90d-63968b6abed5 2026-02-17T12:28:53.195 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:53.187+0000 7f9a2ff318c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:28:53.196 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:53.187+0000 7f9a2ff318c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:28:53.197 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:53.187+0000 7f9a2ff318c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:28:53.198 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:53.187+0000 7f9a2ff318c0 -1 bdev(0x55fe753bfc00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:28:53.198 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:53.187+0000 7f9a2ff318c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T12:28:55.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-17T12:28:55.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:28:55.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:28:55.469 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:28:55.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:28:55.659 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:28:55.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:28:55.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 2026-02-17T12:28:55.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:28:55.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:28:55.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:28:55.684 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:55.675+0000 7ffa49ee18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:28:55.685 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:55.675+0000 7ffa49ee18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:28:55.686 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:55.679+0000 7ffa49ee18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:28:55.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:28:55.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:28:55.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:28:55.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:28:55.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:28:55.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:28:55.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:28:55.816 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:28:55.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:28:55.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:28:55.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:28:56.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:28:56.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:28:56.970 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:28:56.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:28:56.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:28:56.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:28:57.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:28:57.180 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:28:57.171+0000 7ffa49ee18c0 -1 Falling back to public interface 2026-02-17T12:28:58.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:28:58.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:28:58.120 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:28:58.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:28:58.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:28:58.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:28:58.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:28:59.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:28:59.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:28:59.270 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:28:59.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:28:59.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:28:59.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:28:59.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:29:00.176 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:00.167+0000 7ffa49ee18c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:29:00.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:29:00.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:29:00.419 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:29:00.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:29:00.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:29:00.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:29:00.567 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1809438669,v1:127.0.0.1:6811/1809438669] [v2:127.0.0.1:6812/1809438669,v1:127.0.0.1:6813/1809438669] exists,up e1acd1cf-ff5f-4e6f-b90d-63968b6abed5 2026-02-17T12:29:00.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:29:00.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:29:00.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:29:00.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:551: TEST_dump_scrub_schedule: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:29:00.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:553: TEST_dump_scrub_schedule: run_osd td/osd-scrub-test 2 --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 2026-02-17T12:29:00.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:29:00.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:29:00.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:29:00.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:29:00.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-17T12:29:00.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:29:00.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:29:00.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:29:00.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:29:00.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-17T12:29:00.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-17T12:29:00.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:29:00.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:29:00.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:29:00.573 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:29:00.573 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:29:00.573 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:29:00.573 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:29:00.574 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:29:00.574 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:29:00.574 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:29:00.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:29:00.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:29:00.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:29:00.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:29:00.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:29:00.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:29:00.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:29:00.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:29:00.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:29:00.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:29:00.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:29:00.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2' 2026-02-17T12:29:00.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-17T12:29:00.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:29:00.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e0f6eb2a-8f2d-4727-91de-6f8bb779f733 2026-02-17T12:29:00.576 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 e0f6eb2a-8f2d-4727-91de-6f8bb779f733 2026-02-17T12:29:00.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 e0f6eb2a-8f2d-4727-91de-6f8bb779f733' 2026-02-17T12:29:00.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:29:00.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAMX5Rpwo58IhAA9+Ie8WdEjIRU4A9bfD2AgA== 2026-02-17T12:29:00.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAMX5Rpwo58IhAA9+Ie8WdEjIRU4A9bfD2AgA=="}' 2026-02-17T12:29:00.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e0f6eb2a-8f2d-4727-91de-6f8bb779f733 -i td/osd-scrub-test/2/new.json 2026-02-17T12:29:00.735 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:29:00.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-17T12:29:00.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 --mkfs --key AQAMX5Rpwo58IhAA9+Ie8WdEjIRU4A9bfD2AgA== --osd-uuid e0f6eb2a-8f2d-4727-91de-6f8bb779f733 2026-02-17T12:29:00.760 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:00.751+0000 7f99e0cc38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:00.762 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:00.751+0000 7f99e0cc38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:00.763 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:00.755+0000 7f99e0cc38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:00.763 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:00.755+0000 7f99e0cc38c0 -1 bdev(0x56295d50bc00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:29:00.764 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:00.755+0000 7f99e0cc38c0 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-17T12:29:03.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-17T12:29:03.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:29:03.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:29:03.053 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T12:29:03.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:29:03.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:29:03.243 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T12:29:03.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_deep_scrub_randomize_ratio=0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_backoff_ratio=0.0 --osd_op_queue=wpq --osd_stats_update_period_not_scrubbing=1 --osd_stats_update_period_scrubbing=1 --osd_scrub_sleep=0.2 2026-02-17T12:29:03.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:29:03.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:29:03.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:29:03.267 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:03.259+0000 7fac727028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:03.268 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:03.259+0000 7fac727028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:03.269 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:03.259+0000 7fac727028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:03.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:29:03.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:29:03.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:29:03.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:29:03.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:29:03.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:29:03.398 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:29:03.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:29:03.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:29:03.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:29:03.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:29:04.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:29:04.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:29:04.547 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:29:04.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:29:04.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:29:04.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:29:04.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:29:04.760 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:04.751+0000 7fac727028c0 -1 Falling back to public interface 2026-02-17T12:29:05.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:29:05.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:29:05.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:29:05.695 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:29:05.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:29:05.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:29:05.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:29:06.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:29:06.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:29:06.846 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:29:06.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:29:06.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:29:06.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:29:06.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:29:07.766 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:07.755+0000 7fac727028c0 -1 osd.2 0 log_to_monitors true 2026-02-17T12:29:07.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:29:07.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:29:07.996 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:29:07.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:29:07.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:29:07.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:29:08.140 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1081414429,v1:127.0.0.1:6819/1081414429] [v2:127.0.0.1:6820/1081414429,v1:127.0.0.1:6821/1081414429] exists,up e0f6eb2a-8f2d-4727-91de-6f8bb779f733 2026-02-17T12:29:08.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:29:08.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:29:08.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:29:08.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:557: TEST_dump_scrub_schedule: create_pool test 1 1 2026-02-17T12:29:08.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T12:29:08.327 INFO:tasks.workunit.client.0.trial128.stderr:pool 'test' created 2026-02-17T12:29:08.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:29:09.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:558: TEST_dump_scrub_schedule: wait_for_clean 2026-02-17T12:29:09.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:29:09.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:29:09.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:29:09.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:29:09.340 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:29:09.340 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:29:09.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:29:09.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:29:09.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:29:09.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:29:09.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:29:09.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:29:09.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:29:09.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:29:09.446 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:29:09.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:29:09.594 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:29:09.594 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:29:09.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:29:09.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:29:09.595 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:29:09.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836497 2026-02-17T12:29:09.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836497 2026-02-17T12:29:09.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497' 2026-02-17T12:29:09.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:29:09.668 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:29:09.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705675 2026-02-17T12:29:09.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705675 2026-02-17T12:29:09.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-38654705675' 2026-02-17T12:29:09.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:29:09.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:29:09.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542146 2026-02-17T12:29:09.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542146 2026-02-17T12:29:09.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-38654705675 2-60129542146' 2026-02-17T12:29:09.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:29:09.812 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836497 2026-02-17T12:29:09.812 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:29:09.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:29:09.813 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836497 2026-02-17T12:29:09.814 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:29:09.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836497 2026-02-17T12:29:09.815 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836497 2026-02-17T12:29:09.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2026-02-17T12:29:09.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:29:09.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836497 2026-02-17T12:29:09.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:29:10.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:29:10.971 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:29:11.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836497 2026-02-17T12:29:11.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:29:12.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-17T12:29:12.121 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:29:12.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836497 2026-02-17T12:29:12.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:29:12.279 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705675 2026-02-17T12:29:12.279 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:29:12.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:29:12.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705675 2026-02-17T12:29:12.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:29:12.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705675 2026-02-17T12:29:12.282 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705675 2026-02-17T12:29:12.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-17T12:29:12.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:29:12.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705676 -lt 38654705675 2026-02-17T12:29:12.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:29:12.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542146 2026-02-17T12:29:12.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:29:12.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:29:12.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542146 2026-02-17T12:29:12.443 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:29:12.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542146 2026-02-17T12:29:12.444 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542146 2026-02-17T12:29:12.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-17T12:29:12.445 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:29:12.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542148 -lt 60129542146 2026-02-17T12:29:12.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:29:12.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:29:12.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:29:12.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:29:12.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:29:12.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:29:12.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:29:12.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:29:12.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:29:12.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:29:12.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:29:12.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:29:12.933 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:29:12.933 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:29:12.933 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:29:13.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:29:13.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:29:13.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:29:13.111 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:559: TEST_dump_scrub_schedule: ceph osd dump 2026-02-17T12:29:13.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:559: TEST_dump_scrub_schedule: grep '^pool.*['\'']test['\'']' 2026-02-17T12:29:13.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:559: TEST_dump_scrub_schedule: awk '{ print $2 }' 2026-02-17T12:29:13.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:559: TEST_dump_scrub_schedule: poolid=1 2026-02-17T12:29:13.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:561: TEST_dump_scrub_schedule: dd if=/dev/urandom of=testdata.508089 bs=1032 count=1 2026-02-17T12:29:13.267 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records in 2026-02-17T12:29:13.267 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records out 2026-02-17T12:29:13.267 INFO:tasks.workunit.client.0.trial128.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 4.8089e-05 s, 21.5 MB/s 2026-02-17T12:29:13.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: seq 1 90 2026-02-17T12:29:13.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj1 testdata.508089 2026-02-17T12:29:13.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj2 testdata.508089 2026-02-17T12:29:13.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj3 testdata.508089 2026-02-17T12:29:13.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj4 testdata.508089 2026-02-17T12:29:13.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj5 testdata.508089 2026-02-17T12:29:13.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj6 testdata.508089 2026-02-17T12:29:13.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj7 testdata.508089 2026-02-17T12:29:13.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj8 testdata.508089 2026-02-17T12:29:13.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj9 testdata.508089 2026-02-17T12:29:13.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj10 testdata.508089 2026-02-17T12:29:13.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj11 testdata.508089 2026-02-17T12:29:13.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj12 testdata.508089 2026-02-17T12:29:13.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj13 testdata.508089 2026-02-17T12:29:13.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj14 testdata.508089 2026-02-17T12:29:13.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj15 testdata.508089 2026-02-17T12:29:13.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj16 testdata.508089 2026-02-17T12:29:13.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj17 testdata.508089 2026-02-17T12:29:13.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj18 testdata.508089 2026-02-17T12:29:13.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj19 testdata.508089 2026-02-17T12:29:13.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj20 testdata.508089 2026-02-17T12:29:13.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj21 testdata.508089 2026-02-17T12:29:13.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj22 testdata.508089 2026-02-17T12:29:13.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj23 testdata.508089 2026-02-17T12:29:13.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj24 testdata.508089 2026-02-17T12:29:13.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj25 testdata.508089 2026-02-17T12:29:13.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj26 testdata.508089 2026-02-17T12:29:13.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj27 testdata.508089 2026-02-17T12:29:13.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj28 testdata.508089 2026-02-17T12:29:13.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj29 testdata.508089 2026-02-17T12:29:13.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj30 testdata.508089 2026-02-17T12:29:13.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj31 testdata.508089 2026-02-17T12:29:13.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:13.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj32 testdata.508089 2026-02-17T12:29:14.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj33 testdata.508089 2026-02-17T12:29:14.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj34 testdata.508089 2026-02-17T12:29:14.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj35 testdata.508089 2026-02-17T12:29:14.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj36 testdata.508089 2026-02-17T12:29:14.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj37 testdata.508089 2026-02-17T12:29:14.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj38 testdata.508089 2026-02-17T12:29:14.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj39 testdata.508089 2026-02-17T12:29:14.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj40 testdata.508089 2026-02-17T12:29:14.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj41 testdata.508089 2026-02-17T12:29:14.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj42 testdata.508089 2026-02-17T12:29:14.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj43 testdata.508089 2026-02-17T12:29:14.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj44 testdata.508089 2026-02-17T12:29:14.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj45 testdata.508089 2026-02-17T12:29:14.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj46 testdata.508089 2026-02-17T12:29:14.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj47 testdata.508089 2026-02-17T12:29:14.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj48 testdata.508089 2026-02-17T12:29:14.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj49 testdata.508089 2026-02-17T12:29:14.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj50 testdata.508089 2026-02-17T12:29:14.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj51 testdata.508089 2026-02-17T12:29:14.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj52 testdata.508089 2026-02-17T12:29:14.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj53 testdata.508089 2026-02-17T12:29:14.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj54 testdata.508089 2026-02-17T12:29:14.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj55 testdata.508089 2026-02-17T12:29:14.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj56 testdata.508089 2026-02-17T12:29:14.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj57 testdata.508089 2026-02-17T12:29:14.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj58 testdata.508089 2026-02-17T12:29:14.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj59 testdata.508089 2026-02-17T12:29:14.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj60 testdata.508089 2026-02-17T12:29:14.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj61 testdata.508089 2026-02-17T12:29:14.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj62 testdata.508089 2026-02-17T12:29:14.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj63 testdata.508089 2026-02-17T12:29:14.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj64 testdata.508089 2026-02-17T12:29:14.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj65 testdata.508089 2026-02-17T12:29:14.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj66 testdata.508089 2026-02-17T12:29:14.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj67 testdata.508089 2026-02-17T12:29:14.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.835 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj68 testdata.508089 2026-02-17T12:29:14.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj69 testdata.508089 2026-02-17T12:29:14.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj70 testdata.508089 2026-02-17T12:29:14.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj71 testdata.508089 2026-02-17T12:29:14.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj72 testdata.508089 2026-02-17T12:29:14.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj73 testdata.508089 2026-02-17T12:29:14.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:14.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj74 testdata.508089 2026-02-17T12:29:15.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:15.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj75 testdata.508089 2026-02-17T12:29:15.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:15.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj76 testdata.508089 2026-02-17T12:29:15.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:15.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj77 testdata.508089 2026-02-17T12:29:15.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:15.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj78 testdata.508089 2026-02-17T12:29:15.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:15.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj79 testdata.508089 2026-02-17T12:29:15.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:15.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj80 testdata.508089 2026-02-17T12:29:15.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:15.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj81 testdata.508089 2026-02-17T12:29:15.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:15.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj82 testdata.508089 2026-02-17T12:29:15.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:15.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj83 testdata.508089 2026-02-17T12:29:15.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:15.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj84 testdata.508089 2026-02-17T12:29:15.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:15.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj85 testdata.508089 2026-02-17T12:29:15.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:15.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj86 testdata.508089 2026-02-17T12:29:15.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:15.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj87 testdata.508089 2026-02-17T12:29:15.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:15.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj88 testdata.508089 2026-02-17T12:29:15.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:15.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj89 testdata.508089 2026-02-17T12:29:15.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:562: TEST_dump_scrub_schedule: for i in `seq 1 $objects` 2026-02-17T12:29:15.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:564: TEST_dump_scrub_schedule: rados -p test put obj90 testdata.508089 2026-02-17T12:29:15.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:566: TEST_dump_scrub_schedule: rm -f testdata.508089 2026-02-17T12:29:15.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:568: TEST_dump_scrub_schedule: local pgid=1.0 2026-02-17T12:29:15.374 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:570: TEST_dump_scrub_schedule: date +%Y-%m-%dT%H:%M:%S.%N%:z 2026-02-17T12:29:15.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:570: TEST_dump_scrub_schedule: local now_is=2026-02-17T12:29:15.371661422+00:00 2026-02-17T12:29:15.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:578: TEST_dump_scrub_schedule: expct_starting=(['query_active']='false' ['query_is_future']='true' ['query_schedule']='scrub scheduled') 2026-02-17T12:29:15.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:578: TEST_dump_scrub_schedule: declare -A expct_starting 2026-02-17T12:29:15.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:579: TEST_dump_scrub_schedule: declare -A sched_data 2026-02-17T12:29:15.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:580: TEST_dump_scrub_schedule: extract_published_sch 1.0 2026-02-17T12:29:15.371661422+00:00 2019-10-12T20:32:43.645168+0000 sched_data 2026-02-17T12:29:15.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:17: extract_published_sch: local pgn=1.0 2026-02-17T12:29:15.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:18: extract_published_sch: local -n dict=sched_data 2026-02-17T12:29:15.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:19: extract_published_sch: local current_time=2026-02-17T12:29:15.371661422+00:00 2026-02-17T12:29:15.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:20: extract_published_sch: local extra_time=2019-10-12T20:32:43.645168+0000 2026-02-17T12:29:15.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:21: extract_published_sch: local extr_dbg=2 2026-02-17T12:29:15.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:24: extract_published_sch: local saved_echo_flag=x 2026-02-17T12:29:15.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:25: extract_published_sch: set +x 2026-02-17T12:29:15.519 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:29:15.529 INFO:tasks.workunit.client.0.trial128.stdout:from pg dump pg: { 2026-02-17T12:29:15.529 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_pg_state": "active+clean", 2026-02-17T12:29:15.529 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T12:29:15.529 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_last_duration": 0, 2026-02-17T12:29:15.529 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T12:29:15.529 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule_at": "2026-02-18T12:29:08.320641+0000", 2026-02-17T12:29:15.529 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_is_future": true, 2026-02-17T12:29:15.529 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_vs_date": true, 2026-02-17T12:29:15.529 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_reported_epoch": 17, 2026-02-17T12:29:15.529 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_seq": 60 2026-02-17T12:29:15.529 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:15.529 INFO:tasks.workunit.client.0.trial128.stdout:query output: 2026-02-17T12:29:15.599 INFO:tasks.workunit.client.0.trial128.stdout: "scrubber": { 2026-02-17T12:29:15.599 INFO:tasks.workunit.client.0.trial128.stdout: "active": false, 2026-02-17T12:29:15.599 INFO:tasks.workunit.client.0.trial128.stdout: "must_scrub": false, 2026-02-17T12:29:15.599 INFO:tasks.workunit.client.0.trial128.stdout: "must_deep_scrub": false, 2026-02-17T12:29:15.599 INFO:tasks.workunit.client.0.trial128.stdout: "must_repair": false, 2026-02-17T12:29:15.599 INFO:tasks.workunit.client.0.trial128.stdout: "need_auto": false, 2026-02-17T12:29:15.599 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reg_stamp": "2026-02-18T12:29:08.320641+0000", 2026-02-17T12:29:15.599 INFO:tasks.workunit.client.0.trial128.stdout: "schedule": "scrub scheduled @ 2026-02-18T12:29:08.320 (2026-02-18T12:29:08.320)" 2026-02-17T12:29:15.599 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:15.599 INFO:tasks.workunit.client.0.trial128.stdout: "agent_state": {} 2026-02-17T12:29:15.683 INFO:tasks.workunit.client.0.trial128.stdout:( 2026-02-17T12:29:15.683 INFO:tasks.workunit.client.0.trial128.stdout:[query_epoch]=17 2026-02-17T12:29:15.683 INFO:tasks.workunit.client.0.trial128.stdout:[query_seq]=107 2026-02-17T12:29:15.683 INFO:tasks.workunit.client.0.trial128.stdout:[query_active]=false 2026-02-17T12:29:15.683 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule]='scrub scheduled' 2026-02-17T12:29:15.683 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule_at]='2026-02-18T12:29:08.320' 2026-02-17T12:29:15.683 INFO:tasks.workunit.client.0.trial128.stdout:[query_target_at]='2026-02-18T12:29:08.320' 2026-02-17T12:29:15.683 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_duration]=0 2026-02-17T12:29:15.683 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_stamp]='2026-02-17T12:29:08.320641+0000' 2026-02-17T12:29:15.683 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_scrub]='0x0' 2026-02-17T12:29:15.683 INFO:tasks.workunit.client.0.trial128.stdout:[query_is_future]=true 2026-02-17T12:29:15.683 INFO:tasks.workunit.client.0.trial128.stdout:[query_vs_date]=true 2026-02-17T12:29:15.684 INFO:tasks.workunit.client.0.trial128.stdout:[query_scrub_seq]=null 2026-02-17T12:29:15.684 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_pg_state]='active+clean' 2026-02-17T12:29:15.684 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T12:29:15.684 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_last_duration]=0 2026-02-17T12:29:15.684 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T12:29:15.684 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule_at]='2026-02-18T12:29:08.320641+0000' 2026-02-17T12:29:15.684 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_is_future]=true 2026-02-17T12:29:15.684 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_vs_date]=true 2026-02-17T12:29:15.684 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_reported_epoch]=17 2026-02-17T12:29:15.684 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_seq]=60 2026-02-17T12:29:15.685 INFO:tasks.workunit.client.0.trial128.stdout:) 2026-02-17T12:29:15.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:581: TEST_dump_scrub_schedule: schedule_against_expected sched_data expct_starting initial 2026-02-17T12:29:15.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:178: schedule_against_expected: local -n dict=sched_data 2026-02-17T12:29:15.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:179: schedule_against_expected: local -n ep=expct_starting 2026-02-17T12:29:15.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:180: schedule_against_expected: local extr_dbg=1 2026-02-17T12:29:15.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:183: schedule_against_expected: local saved_echo_flag=x 2026-02-17T12:29:15.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:184: schedule_against_expected: set +x 2026-02-17T12:29:15.695 INFO:tasks.workunit.client.0.trial128.stdout:-- - comparing: 2026-02-17T12:29:15.695 INFO:tasks.workunit.client.0.trial128.stdout:key is query_schedule expected: scrub scheduled in actual: scrub scheduled 2026-02-17T12:29:15.695 INFO:tasks.workunit.client.0.trial128.stdout:key is query_is_future expected: true in actual: true 2026-02-17T12:29:15.695 INFO:tasks.workunit.client.0.trial128.stdout:key is query_active expected: false in actual: false 2026-02-17T12:29:15.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:203: schedule_against_expected: return 0 2026-02-17T12:29:15.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:582: TEST_dump_scrub_schedule: (( 0 == 0 )) 2026-02-17T12:29:15.695 INFO:tasks.workunit.client.0.trial128.stdout:last-scrub --- 0x0 2026-02-17T12:29:15.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:583: TEST_dump_scrub_schedule: echo 'last-scrub --- ' 0x0 2026-02-17T12:29:15.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:589: TEST_dump_scrub_schedule: saved_last_stamp=2026-02-17T12:29:08.320641+0000 2026-02-17T12:29:15.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:590: TEST_dump_scrub_schedule: ceph tell 'osd.*' config set osd_scrub_sleep 0 2026-02-17T12:29:15.757 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T12:29:15.757 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_sleep = '' " 2026-02-17T12:29:15.757 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:15.763 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T12:29:15.763 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_sleep = '' " 2026-02-17T12:29:15.764 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:15.770 INFO:tasks.workunit.client.0.trial128.stdout:osd.2: { 2026-02-17T12:29:15.770 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_sleep = '' " 2026-02-17T12:29:15.770 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:15.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:591: TEST_dump_scrub_schedule: ceph tell 1.0 deep-scrub 2026-02-17T12:29:15.851 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:29:15.852 INFO:tasks.workunit.client.0.trial128.stdout: "deep": true, 2026-02-17T12:29:15.852 INFO:tasks.workunit.client.0.trial128.stdout: "must": true, 2026-02-17T12:29:15.852 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "0.000000" 2026-02-17T12:29:15.852 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:15.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:596: TEST_dump_scrub_schedule: sleep 5 2026-02-17T12:29:20.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:597: TEST_dump_scrub_schedule: sched_data=() 2026-02-17T12:29:20.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:598: TEST_dump_scrub_schedule: expct_qry_duration=(['query_last_duration']='0' ['query_last_duration_neg']='not0') 2026-02-17T12:29:20.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:598: TEST_dump_scrub_schedule: declare -A expct_qry_duration 2026-02-17T12:29:20.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:599: TEST_dump_scrub_schedule: wait_any_cond 1.0 10 2026-02-17T12:29:08.320641+0000 expct_qry_duration 'WaitingAfterScrub ' sched_data 2026-02-17T12:29:20.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.0 2026-02-17T12:29:20.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-17T12:29:20.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-17T12:29:08.320641+0000 2026-02-17T12:29:20.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=expct_qry_duration 2026-02-17T12:29:20.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sched_data 2026-02-17T12:29:20.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-17T12:29:20.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-17T12:29:20.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-17T12:29:20.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-17T12:29:20.862 INFO:tasks.workunit.client.0.trial128.stdout:waiting for any condition (WaitingAfterScrub ): pg:1.0 dt:2026-02-17T12:29:08.320641+0000 (10 retries) 2026-02-17T12:29:21.505 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:29:21.514 INFO:tasks.workunit.client.0.trial128.stdout:from pg dump pg: { 2026-02-17T12:29:21.514 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_pg_state": "active+clean", 2026-02-17T12:29:21.515 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T12:29:21.515 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_last_duration": 1, 2026-02-17T12:29:21.515 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T12:29:21.515 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule_at": "2026-02-18T12:29:16.147066+0000", 2026-02-17T12:29:21.515 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_is_future": true, 2026-02-17T12:29:21.515 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_vs_date": true, 2026-02-17T12:29:21.515 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_reported_epoch": 17, 2026-02-17T12:29:21.515 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_seq": 124 2026-02-17T12:29:21.516 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:21.516 INFO:tasks.workunit.client.0.trial128.stdout:query output: 2026-02-17T12:29:21.585 INFO:tasks.workunit.client.0.trial128.stdout: "scrubber": { 2026-02-17T12:29:21.585 INFO:tasks.workunit.client.0.trial128.stdout: "active": false, 2026-02-17T12:29:21.585 INFO:tasks.workunit.client.0.trial128.stdout: "must_scrub": false, 2026-02-17T12:29:21.585 INFO:tasks.workunit.client.0.trial128.stdout: "must_deep_scrub": false, 2026-02-17T12:29:21.585 INFO:tasks.workunit.client.0.trial128.stdout: "must_repair": false, 2026-02-17T12:29:21.585 INFO:tasks.workunit.client.0.trial128.stdout: "need_auto": false, 2026-02-17T12:29:21.585 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reg_stamp": "2026-02-18T12:29:16.147066+0000", 2026-02-17T12:29:21.585 INFO:tasks.workunit.client.0.trial128.stdout: "schedule": "scrub scheduled @ 2026-02-18T12:29:16.147 (2026-02-18T12:29:16.147)" 2026-02-17T12:29:21.586 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:21.586 INFO:tasks.workunit.client.0.trial128.stdout: "agent_state": {} 2026-02-17T12:29:21.665 INFO:tasks.workunit.client.0.trial128.stdout:( 2026-02-17T12:29:21.666 INFO:tasks.workunit.client.0.trial128.stdout:[query_epoch]=17 2026-02-17T12:29:21.666 INFO:tasks.workunit.client.0.trial128.stdout:[query_seq]=125 2026-02-17T12:29:21.666 INFO:tasks.workunit.client.0.trial128.stdout:[query_active]=false 2026-02-17T12:29:21.666 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule]='scrub scheduled' 2026-02-17T12:29:21.666 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule_at]='2026-02-18T12:29:16.147' 2026-02-17T12:29:21.666 INFO:tasks.workunit.client.0.trial128.stdout:[query_target_at]='2026-02-18T12:29:16.147' 2026-02-17T12:29:21.666 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_duration]=1 2026-02-17T12:29:21.666 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_stamp]='2026-02-17T12:29:16.147066+0000' 2026-02-17T12:29:21.666 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_scrub]='17x90' 2026-02-17T12:29:21.667 INFO:tasks.workunit.client.0.trial128.stdout:[query_is_future]=true 2026-02-17T12:29:21.667 INFO:tasks.workunit.client.0.trial128.stdout:[query_vs_date]=true 2026-02-17T12:29:21.667 INFO:tasks.workunit.client.0.trial128.stdout:[query_scrub_seq]=null 2026-02-17T12:29:21.667 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_pg_state]='active+clean' 2026-02-17T12:29:21.667 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T12:29:21.667 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_last_duration]=1 2026-02-17T12:29:21.667 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T12:29:21.667 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule_at]='2026-02-18T12:29:16.147066+0000' 2026-02-17T12:29:21.667 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_is_future]=true 2026-02-17T12:29:21.667 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_vs_date]=true 2026-02-17T12:29:21.667 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_reported_epoch]=17 2026-02-17T12:29:21.667 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_seq]=124 2026-02-17T12:29:21.668 INFO:tasks.workunit.client.0.trial128.stdout:) 2026-02-17T12:29:21.678 INFO:tasks.workunit.client.0.trial128.stdout:--> loop: 1 ~ false / 125 / 124 / true / 2026-02-17T12:29:16.147066+0000 / scrub scheduled %%% query_last_duration_neg query_last_duration 2026-02-17T12:29:21.678 INFO:tasks.workunit.client.0.trial128.stdout:key is query_last_duration: negation:1 # expected: 0 # in actual: 1 2026-02-17T12:29:21.678 INFO:tasks.workunit.client.0.trial128.stdout:WaitingAfterScrub - 'query_last_duration' actual value (1) matches expected (0) (negation: 1) 2026-02-17T12:29:21.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-17T12:29:21.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:601: TEST_dump_scrub_schedule: sched_data=() 2026-02-17T12:29:21.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:602: TEST_dump_scrub_schedule: expct_dmp_duration=(['dmp_last_duration']='0' ['dmp_last_duration_neg']='not0') 2026-02-17T12:29:21.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:602: TEST_dump_scrub_schedule: declare -A expct_dmp_duration 2026-02-17T12:29:21.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:603: TEST_dump_scrub_schedule: wait_any_cond 1.0 10 2026-02-17T12:29:08.320641+0000 expct_dmp_duration 'WaitingAfterScrub_dmp ' sched_data 2026-02-17T12:29:21.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.0 2026-02-17T12:29:21.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-17T12:29:21.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-17T12:29:08.320641+0000 2026-02-17T12:29:21.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=expct_dmp_duration 2026-02-17T12:29:21.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sched_data 2026-02-17T12:29:21.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-17T12:29:21.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-17T12:29:21.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-17T12:29:21.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-17T12:29:21.681 INFO:tasks.workunit.client.0.trial128.stdout:waiting for any condition (WaitingAfterScrub_dmp ): pg:1.0 dt:2026-02-17T12:29:08.320641+0000 (10 retries) 2026-02-17T12:29:22.329 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:29:22.339 INFO:tasks.workunit.client.0.trial128.stdout:from pg dump pg: { 2026-02-17T12:29:22.339 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_pg_state": "active+clean", 2026-02-17T12:29:22.339 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T12:29:22.339 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_last_duration": 1, 2026-02-17T12:29:22.339 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T12:29:22.339 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule_at": "2026-02-18T12:29:16.147066+0000", 2026-02-17T12:29:22.339 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_is_future": true, 2026-02-17T12:29:22.339 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_vs_date": true, 2026-02-17T12:29:22.339 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_reported_epoch": 17, 2026-02-17T12:29:22.339 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_seq": 124 2026-02-17T12:29:22.339 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:22.339 INFO:tasks.workunit.client.0.trial128.stdout:query output: 2026-02-17T12:29:22.409 INFO:tasks.workunit.client.0.trial128.stdout: "scrubber": { 2026-02-17T12:29:22.409 INFO:tasks.workunit.client.0.trial128.stdout: "active": false, 2026-02-17T12:29:22.409 INFO:tasks.workunit.client.0.trial128.stdout: "must_scrub": false, 2026-02-17T12:29:22.409 INFO:tasks.workunit.client.0.trial128.stdout: "must_deep_scrub": false, 2026-02-17T12:29:22.409 INFO:tasks.workunit.client.0.trial128.stdout: "must_repair": false, 2026-02-17T12:29:22.409 INFO:tasks.workunit.client.0.trial128.stdout: "need_auto": false, 2026-02-17T12:29:22.409 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reg_stamp": "2026-02-18T12:29:16.147066+0000", 2026-02-17T12:29:22.409 INFO:tasks.workunit.client.0.trial128.stdout: "schedule": "scrub scheduled @ 2026-02-18T12:29:16.147 (2026-02-18T12:29:16.147)" 2026-02-17T12:29:22.409 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:22.409 INFO:tasks.workunit.client.0.trial128.stdout: "agent_state": {} 2026-02-17T12:29:22.490 INFO:tasks.workunit.client.0.trial128.stdout:( 2026-02-17T12:29:22.490 INFO:tasks.workunit.client.0.trial128.stdout:[query_epoch]=17 2026-02-17T12:29:22.490 INFO:tasks.workunit.client.0.trial128.stdout:[query_seq]=126 2026-02-17T12:29:22.491 INFO:tasks.workunit.client.0.trial128.stdout:[query_active]=false 2026-02-17T12:29:22.491 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule]='scrub scheduled' 2026-02-17T12:29:22.491 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule_at]='2026-02-18T12:29:16.147' 2026-02-17T12:29:22.491 INFO:tasks.workunit.client.0.trial128.stdout:[query_target_at]='2026-02-18T12:29:16.147' 2026-02-17T12:29:22.491 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_duration]=1 2026-02-17T12:29:22.491 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_stamp]='2026-02-17T12:29:16.147066+0000' 2026-02-17T12:29:22.491 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_scrub]='17x90' 2026-02-17T12:29:22.491 INFO:tasks.workunit.client.0.trial128.stdout:[query_is_future]=true 2026-02-17T12:29:22.491 INFO:tasks.workunit.client.0.trial128.stdout:[query_vs_date]=true 2026-02-17T12:29:22.491 INFO:tasks.workunit.client.0.trial128.stdout:[query_scrub_seq]=null 2026-02-17T12:29:22.491 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_pg_state]='active+clean' 2026-02-17T12:29:22.491 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T12:29:22.491 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_last_duration]=1 2026-02-17T12:29:22.492 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T12:29:22.492 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule_at]='2026-02-18T12:29:16.147066+0000' 2026-02-17T12:29:22.492 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_is_future]=true 2026-02-17T12:29:22.492 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_vs_date]=true 2026-02-17T12:29:22.492 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_reported_epoch]=17 2026-02-17T12:29:22.492 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_seq]=124 2026-02-17T12:29:22.492 INFO:tasks.workunit.client.0.trial128.stdout:) 2026-02-17T12:29:22.500 INFO:tasks.workunit.client.0.trial128.stdout:--> loop: 1 ~ false / 126 / 124 / true / 2026-02-17T12:29:16.147066+0000 / scrub scheduled %%% dmp_last_duration dmp_last_duration_neg 2026-02-17T12:29:22.500 INFO:tasks.workunit.client.0.trial128.stdout:key is dmp_last_duration: negation:1 # expected: 0 # in actual: 1 2026-02-17T12:29:22.501 INFO:tasks.workunit.client.0.trial128.stdout:WaitingAfterScrub_dmp - 'dmp_last_duration' actual value (1) matches expected (0) (negation: 1) 2026-02-17T12:29:22.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-17T12:29:22.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:610: TEST_dump_scrub_schedule: ceph osd set noscrub 2026-02-17T12:29:22.700 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:29:22.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:611: TEST_dump_scrub_schedule: sleep 2 2026-02-17T12:29:24.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:612: TEST_dump_scrub_schedule: ceph tell 'osd.*' config set osd_shallow_scrub_chunk_max 3 2026-02-17T12:29:24.778 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T12:29:24.778 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_shallow_scrub_chunk_max = '' (not observed, change may require restart) " 2026-02-17T12:29:24.778 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:24.784 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T12:29:24.784 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_shallow_scrub_chunk_max = '' (not observed, change may require restart) " 2026-02-17T12:29:24.784 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:24.790 INFO:tasks.workunit.client.0.trial128.stdout:osd.2: { 2026-02-17T12:29:24.790 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_shallow_scrub_chunk_max = '' (not observed, change may require restart) " 2026-02-17T12:29:24.790 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:24.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:613: TEST_dump_scrub_schedule: ceph tell 'osd.*' config set osd_scrub_sleep 2.0 2026-02-17T12:29:24.860 INFO:tasks.workunit.client.0.trial128.stdout:osd.0: { 2026-02-17T12:29:24.860 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_sleep = '' " 2026-02-17T12:29:24.860 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:24.867 INFO:tasks.workunit.client.0.trial128.stdout:osd.1: { 2026-02-17T12:29:24.867 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_sleep = '' " 2026-02-17T12:29:24.867 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:24.873 INFO:tasks.workunit.client.0.trial128.stdout:osd.2: { 2026-02-17T12:29:24.874 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_sleep = '' " 2026-02-17T12:29:24.874 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:24.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:614: TEST_dump_scrub_schedule: sleep 8 2026-02-17T12:29:32.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:615: TEST_dump_scrub_schedule: saved_last_stamp=2026-02-17T12:29:16.147066+0000 2026-02-17T12:29:32.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:617: TEST_dump_scrub_schedule: ceph tell 1.0 schedule-scrub 2026-02-17T12:29:32.948 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:29:32.949 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:29:32.949 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:29:32.949 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-10T12:27:52.944114+0000" 2026-02-17T12:29:32.949 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:32.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:618: TEST_dump_scrub_schedule: sleep 1 2026-02-17T12:29:33.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:619: TEST_dump_scrub_schedule: sched_data=() 2026-02-17T12:29:33.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:621: TEST_dump_scrub_schedule: expct_scrub_peri_sched=(['query_is_future']='false') 2026-02-17T12:29:33.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:621: TEST_dump_scrub_schedule: declare -A expct_scrub_peri_sched 2026-02-17T12:29:33.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:622: TEST_dump_scrub_schedule: wait_any_cond 1.0 10 2026-02-17T12:29:16.147066+0000 expct_scrub_peri_sched waitingBeingScheduled sched_data 2026-02-17T12:29:33.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.0 2026-02-17T12:29:33.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-17T12:29:33.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-17T12:29:16.147066+0000 2026-02-17T12:29:33.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=expct_scrub_peri_sched 2026-02-17T12:29:33.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sched_data 2026-02-17T12:29:33.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-17T12:29:33.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-17T12:29:33.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-17T12:29:33.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-17T12:29:33.960 INFO:tasks.workunit.client.0.trial128.stdout:waiting for any condition (waitingBeingScheduled): pg:1.0 dt:2026-02-17T12:29:16.147066+0000 (10 retries) 2026-02-17T12:29:34.607 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:29:34.618 INFO:tasks.workunit.client.0.trial128.stdout:from pg dump pg: { 2026-02-17T12:29:34.618 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_pg_state": "active+clean", 2026-02-17T12:29:34.618 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T12:29:34.618 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_last_duration": 1, 2026-02-17T12:29:34.619 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T12:29:34.619 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule_at": "2026-02-18T12:29:16.147066+0000", 2026-02-17T12:29:34.619 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_is_future": true, 2026-02-17T12:29:34.619 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_vs_date": true, 2026-02-17T12:29:34.619 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_reported_epoch": 18, 2026-02-17T12:29:34.619 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_seq": 132 2026-02-17T12:29:34.619 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:34.619 INFO:tasks.workunit.client.0.trial128.stdout:query output: 2026-02-17T12:29:34.689 INFO:tasks.workunit.client.0.trial128.stdout: "scrubber": { 2026-02-17T12:29:34.690 INFO:tasks.workunit.client.0.trial128.stdout: "active": false, 2026-02-17T12:29:34.690 INFO:tasks.workunit.client.0.trial128.stdout: "must_scrub": false, 2026-02-17T12:29:34.690 INFO:tasks.workunit.client.0.trial128.stdout: "must_deep_scrub": false, 2026-02-17T12:29:34.690 INFO:tasks.workunit.client.0.trial128.stdout: "must_repair": false, 2026-02-17T12:29:34.691 INFO:tasks.workunit.client.0.trial128.stdout: "need_auto": false, 2026-02-17T12:29:34.691 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reg_stamp": "2026-02-17T12:29:38.238055+0000", 2026-02-17T12:29:34.691 INFO:tasks.workunit.client.0.trial128.stdout: "schedule": "scrub scheduled @ 2026-02-17T12:29:38.238 (2026-02-11T12:27:52.944)" 2026-02-17T12:29:34.691 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:34.691 INFO:tasks.workunit.client.0.trial128.stdout: "agent_state": {} 2026-02-17T12:29:34.773 INFO:tasks.workunit.client.0.trial128.stdout:( 2026-02-17T12:29:34.773 INFO:tasks.workunit.client.0.trial128.stdout:[query_epoch]=18 2026-02-17T12:29:34.773 INFO:tasks.workunit.client.0.trial128.stdout:[query_seq]=136 2026-02-17T12:29:34.773 INFO:tasks.workunit.client.0.trial128.stdout:[query_active]=false 2026-02-17T12:29:34.773 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule]='scrub scheduled' 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule_at]='2026-02-17T12:29:38.238' 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[query_target_at]='2026-02-11T12:27:52.944' 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_duration]=1 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_stamp]='2026-02-10T12:27:52.944114+0000' 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_scrub]='17x90' 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[query_is_future]=false 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[query_vs_date]=true 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[query_scrub_seq]=null 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_pg_state]='active+clean' 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_last_duration]=1 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule_at]='2026-02-18T12:29:16.147066+0000' 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_is_future]=true 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_vs_date]=true 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_reported_epoch]=18 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_seq]=132 2026-02-17T12:29:34.774 INFO:tasks.workunit.client.0.trial128.stdout:) 2026-02-17T12:29:34.786 INFO:tasks.workunit.client.0.trial128.stdout:--> loop: 1 ~ false / 136 / 132 / false / 2026-02-10T12:27:52.944114+0000 / scrub scheduled %%% query_is_future 2026-02-17T12:29:34.786 INFO:tasks.workunit.client.0.trial128.stdout:key is query_is_future: negation:0 # expected: false # in actual: false 2026-02-17T12:29:34.787 INFO:tasks.workunit.client.0.trial128.stdout:waitingBeingScheduled - 'query_is_future' actual value (false) matches expected (false) (negation: 0) 2026-02-17T12:29:34.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-17T12:29:34.787 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:634: TEST_dump_scrub_schedule: saved_last_stamp=2026-02-10T12:27:52.944114+0000 2026-02-17T12:29:34.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:635: TEST_dump_scrub_schedule: ceph osd unset noscrub 2026-02-17T12:29:34.984 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is unset 2026-02-17T12:29:34.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:637: TEST_dump_scrub_schedule: cond_active=(['query_active']='true') 2026-02-17T12:29:34.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:637: TEST_dump_scrub_schedule: declare -A cond_active 2026-02-17T12:29:34.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:638: TEST_dump_scrub_schedule: sched_data=() 2026-02-17T12:29:34.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:639: TEST_dump_scrub_schedule: wait_any_cond 1.0 10 2026-02-10T12:27:52.944114+0000 cond_active 'WaitingActive ' sched_data 2026-02-17T12:29:34.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.0 2026-02-17T12:29:34.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-17T12:29:34.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-10T12:27:52.944114+0000 2026-02-17T12:29:34.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=cond_active 2026-02-17T12:29:34.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sched_data 2026-02-17T12:29:34.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-17T12:29:34.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-17T12:29:34.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-17T12:29:34.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-17T12:29:34.998 INFO:tasks.workunit.client.0.trial128.stdout:waiting for any condition (WaitingActive ): pg:1.0 dt:2026-02-10T12:27:52.944114+0000 (10 retries) 2026-02-17T12:29:35.642 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:29:35.651 INFO:tasks.workunit.client.0.trial128.stdout:from pg dump pg: { 2026-02-17T12:29:35.651 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_pg_state": "active+clean", 2026-02-17T12:29:35.651 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T12:29:35.652 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_last_duration": 1, 2026-02-17T12:29:35.652 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T12:29:35.652 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule_at": "2026-02-17T12:29:38.238055+0000", 2026-02-17T12:29:35.652 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_is_future": true, 2026-02-17T12:29:35.652 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_vs_date": true, 2026-02-17T12:29:35.652 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_reported_epoch": 18, 2026-02-17T12:29:35.652 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_seq": 135 2026-02-17T12:29:35.652 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:35.652 INFO:tasks.workunit.client.0.trial128.stdout:query output: 2026-02-17T12:29:35.724 INFO:tasks.workunit.client.0.trial128.stdout: "scrubber": { 2026-02-17T12:29:35.725 INFO:tasks.workunit.client.0.trial128.stdout: "active": false, 2026-02-17T12:29:35.725 INFO:tasks.workunit.client.0.trial128.stdout: "must_scrub": false, 2026-02-17T12:29:35.725 INFO:tasks.workunit.client.0.trial128.stdout: "must_deep_scrub": false, 2026-02-17T12:29:35.725 INFO:tasks.workunit.client.0.trial128.stdout: "must_repair": false, 2026-02-17T12:29:35.725 INFO:tasks.workunit.client.0.trial128.stdout: "need_auto": false, 2026-02-17T12:29:35.725 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reg_stamp": "2026-02-17T12:29:38.238055+0000", 2026-02-17T12:29:35.725 INFO:tasks.workunit.client.0.trial128.stdout: "schedule": "scrub scheduled @ 2026-02-17T12:29:38.238 (2026-02-11T12:27:52.944)" 2026-02-17T12:29:35.725 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:35.725 INFO:tasks.workunit.client.0.trial128.stdout: "agent_state": {} 2026-02-17T12:29:35.805 INFO:tasks.workunit.client.0.trial128.stdout:( 2026-02-17T12:29:35.806 INFO:tasks.workunit.client.0.trial128.stdout:[query_epoch]=18 2026-02-17T12:29:35.806 INFO:tasks.workunit.client.0.trial128.stdout:[query_seq]=137 2026-02-17T12:29:35.806 INFO:tasks.workunit.client.0.trial128.stdout:[query_active]=false 2026-02-17T12:29:35.806 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule]='scrub scheduled' 2026-02-17T12:29:35.806 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule_at]='2026-02-17T12:29:38.238' 2026-02-17T12:29:35.806 INFO:tasks.workunit.client.0.trial128.stdout:[query_target_at]='2026-02-11T12:27:52.944' 2026-02-17T12:29:35.806 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_duration]=1 2026-02-17T12:29:35.806 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_stamp]='2026-02-10T12:27:52.944114+0000' 2026-02-17T12:29:35.806 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_scrub]='17x90' 2026-02-17T12:29:35.806 INFO:tasks.workunit.client.0.trial128.stdout:[query_is_future]=false 2026-02-17T12:29:35.807 INFO:tasks.workunit.client.0.trial128.stdout:[query_vs_date]=true 2026-02-17T12:29:35.807 INFO:tasks.workunit.client.0.trial128.stdout:[query_scrub_seq]=null 2026-02-17T12:29:35.807 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_pg_state]='active+clean' 2026-02-17T12:29:35.807 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T12:29:35.807 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_last_duration]=1 2026-02-17T12:29:35.807 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T12:29:35.807 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule_at]='2026-02-17T12:29:38.238055+0000' 2026-02-17T12:29:35.807 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_is_future]=true 2026-02-17T12:29:35.807 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_vs_date]=true 2026-02-17T12:29:35.807 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_reported_epoch]=18 2026-02-17T12:29:35.807 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_seq]=135 2026-02-17T12:29:35.807 INFO:tasks.workunit.client.0.trial128.stdout:) 2026-02-17T12:29:35.817 INFO:tasks.workunit.client.0.trial128.stdout:--> loop: 1 ~ false / 137 / 135 / false / 2026-02-10T12:27:52.944114+0000 / scrub scheduled %%% query_active 2026-02-17T12:29:35.817 INFO:tasks.workunit.client.0.trial128.stdout:key is query_active: negation:0 # expected: true # in actual: false 2026-02-17T12:29:36.464 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:29:36.474 INFO:tasks.workunit.client.0.trial128.stdout:from pg dump pg: { 2026-02-17T12:29:36.474 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_pg_state": "active+clean", 2026-02-17T12:29:36.474 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T12:29:36.474 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_last_duration": 1, 2026-02-17T12:29:36.474 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T12:29:36.474 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule_at": "2026-02-17T12:29:38.238055+0000", 2026-02-17T12:29:36.474 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_is_future": true, 2026-02-17T12:29:36.474 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_vs_date": true, 2026-02-17T12:29:36.474 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_reported_epoch": 18, 2026-02-17T12:29:36.475 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_seq": 135 2026-02-17T12:29:36.475 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:36.475 INFO:tasks.workunit.client.0.trial128.stdout:query output: 2026-02-17T12:29:36.544 INFO:tasks.workunit.client.0.trial128.stdout: "scrubber": { 2026-02-17T12:29:36.544 INFO:tasks.workunit.client.0.trial128.stdout: "active": false, 2026-02-17T12:29:36.544 INFO:tasks.workunit.client.0.trial128.stdout: "must_scrub": false, 2026-02-17T12:29:36.544 INFO:tasks.workunit.client.0.trial128.stdout: "must_deep_scrub": false, 2026-02-17T12:29:36.544 INFO:tasks.workunit.client.0.trial128.stdout: "must_repair": false, 2026-02-17T12:29:36.544 INFO:tasks.workunit.client.0.trial128.stdout: "need_auto": false, 2026-02-17T12:29:36.545 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reg_stamp": "2026-02-17T12:29:38.238055+0000", 2026-02-17T12:29:36.545 INFO:tasks.workunit.client.0.trial128.stdout: "schedule": "scrub scheduled @ 2026-02-17T12:29:38.238 (2026-02-11T12:27:52.944)" 2026-02-17T12:29:36.545 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:36.545 INFO:tasks.workunit.client.0.trial128.stdout: "agent_state": {} 2026-02-17T12:29:36.627 INFO:tasks.workunit.client.0.trial128.stdout:( 2026-02-17T12:29:36.627 INFO:tasks.workunit.client.0.trial128.stdout:[query_epoch]=18 2026-02-17T12:29:36.627 INFO:tasks.workunit.client.0.trial128.stdout:[query_seq]=137 2026-02-17T12:29:36.627 INFO:tasks.workunit.client.0.trial128.stdout:[query_active]=false 2026-02-17T12:29:36.627 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule]='scrub scheduled' 2026-02-17T12:29:36.627 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule_at]='2026-02-17T12:29:38.238' 2026-02-17T12:29:36.627 INFO:tasks.workunit.client.0.trial128.stdout:[query_target_at]='2026-02-11T12:27:52.944' 2026-02-17T12:29:36.627 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_duration]=1 2026-02-17T12:29:36.628 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_stamp]='2026-02-10T12:27:52.944114+0000' 2026-02-17T12:29:36.628 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_scrub]='17x90' 2026-02-17T12:29:36.628 INFO:tasks.workunit.client.0.trial128.stdout:[query_is_future]=false 2026-02-17T12:29:36.628 INFO:tasks.workunit.client.0.trial128.stdout:[query_vs_date]=true 2026-02-17T12:29:36.628 INFO:tasks.workunit.client.0.trial128.stdout:[query_scrub_seq]=null 2026-02-17T12:29:36.628 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_pg_state]='active+clean' 2026-02-17T12:29:36.628 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T12:29:36.628 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_last_duration]=1 2026-02-17T12:29:36.628 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T12:29:36.628 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule_at]='2026-02-17T12:29:38.238055+0000' 2026-02-17T12:29:36.628 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_is_future]=true 2026-02-17T12:29:36.628 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_vs_date]=true 2026-02-17T12:29:36.629 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_reported_epoch]=18 2026-02-17T12:29:36.629 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_seq]=135 2026-02-17T12:29:36.629 INFO:tasks.workunit.client.0.trial128.stdout:) 2026-02-17T12:29:36.638 INFO:tasks.workunit.client.0.trial128.stdout:--> loop: 2 ~ false / 137 / 135 / false / 2026-02-10T12:27:52.944114+0000 / scrub scheduled %%% query_active 2026-02-17T12:29:36.638 INFO:tasks.workunit.client.0.trial128.stdout:key is query_active: negation:0 # expected: true # in actual: false 2026-02-17T12:29:37.285 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:29:37.297 INFO:tasks.workunit.client.0.trial128.stdout:from pg dump pg: { 2026-02-17T12:29:37.297 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_pg_state": "active+clean", 2026-02-17T12:29:37.297 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T12:29:37.297 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_last_duration": 1, 2026-02-17T12:29:37.297 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T12:29:37.297 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule_at": "2026-02-17T12:29:38.238055+0000", 2026-02-17T12:29:37.297 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_is_future": true, 2026-02-17T12:29:37.297 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_vs_date": true, 2026-02-17T12:29:37.297 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_reported_epoch": 18, 2026-02-17T12:29:37.297 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_seq": 136 2026-02-17T12:29:37.298 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:37.298 INFO:tasks.workunit.client.0.trial128.stdout:query output: 2026-02-17T12:29:37.368 INFO:tasks.workunit.client.0.trial128.stdout: "scrubber": { 2026-02-17T12:29:37.368 INFO:tasks.workunit.client.0.trial128.stdout: "active": false, 2026-02-17T12:29:37.368 INFO:tasks.workunit.client.0.trial128.stdout: "must_scrub": false, 2026-02-17T12:29:37.368 INFO:tasks.workunit.client.0.trial128.stdout: "must_deep_scrub": false, 2026-02-17T12:29:37.368 INFO:tasks.workunit.client.0.trial128.stdout: "must_repair": false, 2026-02-17T12:29:37.369 INFO:tasks.workunit.client.0.trial128.stdout: "need_auto": false, 2026-02-17T12:29:37.369 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reg_stamp": "2026-02-17T12:29:38.238055+0000", 2026-02-17T12:29:37.369 INFO:tasks.workunit.client.0.trial128.stdout: "schedule": "scrub scheduled @ 2026-02-17T12:29:38.238 (2026-02-11T12:27:52.944)" 2026-02-17T12:29:37.369 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:37.369 INFO:tasks.workunit.client.0.trial128.stdout: "agent_state": {} 2026-02-17T12:29:37.450 INFO:tasks.workunit.client.0.trial128.stdout:( 2026-02-17T12:29:37.451 INFO:tasks.workunit.client.0.trial128.stdout:[query_epoch]=19 2026-02-17T12:29:37.451 INFO:tasks.workunit.client.0.trial128.stdout:[query_seq]=140 2026-02-17T12:29:37.451 INFO:tasks.workunit.client.0.trial128.stdout:[query_active]=false 2026-02-17T12:29:37.451 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule]='scrub scheduled' 2026-02-17T12:29:37.451 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule_at]='2026-02-17T12:29:38.238' 2026-02-17T12:29:37.451 INFO:tasks.workunit.client.0.trial128.stdout:[query_target_at]='2026-02-11T12:27:52.944' 2026-02-17T12:29:37.451 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_duration]=1 2026-02-17T12:29:37.451 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_stamp]='2026-02-10T12:27:52.944114+0000' 2026-02-17T12:29:37.451 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_scrub]='17x90' 2026-02-17T12:29:37.452 INFO:tasks.workunit.client.0.trial128.stdout:[query_is_future]=false 2026-02-17T12:29:37.452 INFO:tasks.workunit.client.0.trial128.stdout:[query_vs_date]=true 2026-02-17T12:29:37.452 INFO:tasks.workunit.client.0.trial128.stdout:[query_scrub_seq]=null 2026-02-17T12:29:37.452 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_pg_state]='active+clean' 2026-02-17T12:29:37.452 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T12:29:37.452 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_last_duration]=1 2026-02-17T12:29:37.452 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T12:29:37.452 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule_at]='2026-02-17T12:29:38.238055+0000' 2026-02-17T12:29:37.452 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_is_future]=true 2026-02-17T12:29:37.452 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_vs_date]=true 2026-02-17T12:29:37.452 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_reported_epoch]=18 2026-02-17T12:29:37.453 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_seq]=136 2026-02-17T12:29:37.453 INFO:tasks.workunit.client.0.trial128.stdout:) 2026-02-17T12:29:37.462 INFO:tasks.workunit.client.0.trial128.stdout:--> loop: 3 ~ false / 140 / 136 / false / 2026-02-10T12:27:52.944114+0000 / scrub scheduled %%% query_active 2026-02-17T12:29:37.462 INFO:tasks.workunit.client.0.trial128.stdout:key is query_active: negation:0 # expected: true # in actual: false 2026-02-17T12:29:38.104 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:29:38.114 INFO:tasks.workunit.client.0.trial128.stdout:from pg dump pg: { 2026-02-17T12:29:38.114 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_pg_state": "active+clean", 2026-02-17T12:29:38.114 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T12:29:38.114 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_last_duration": 1, 2026-02-17T12:29:38.114 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T12:29:38.114 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule_at": "2026-02-17T12:29:38.238055+0000", 2026-02-17T12:29:38.114 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_is_future": true, 2026-02-17T12:29:38.115 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_vs_date": true, 2026-02-17T12:29:38.115 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_reported_epoch": 18, 2026-02-17T12:29:38.115 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_seq": 136 2026-02-17T12:29:38.115 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:38.115 INFO:tasks.workunit.client.0.trial128.stdout:query output: 2026-02-17T12:29:38.185 INFO:tasks.workunit.client.0.trial128.stdout: "scrubber": { 2026-02-17T12:29:38.185 INFO:tasks.workunit.client.0.trial128.stdout: "active": false, 2026-02-17T12:29:38.185 INFO:tasks.workunit.client.0.trial128.stdout: "must_scrub": false, 2026-02-17T12:29:38.185 INFO:tasks.workunit.client.0.trial128.stdout: "must_deep_scrub": false, 2026-02-17T12:29:38.185 INFO:tasks.workunit.client.0.trial128.stdout: "must_repair": false, 2026-02-17T12:29:38.185 INFO:tasks.workunit.client.0.trial128.stdout: "need_auto": false, 2026-02-17T12:29:38.185 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reg_stamp": "2026-02-17T12:29:38.238055+0000", 2026-02-17T12:29:38.186 INFO:tasks.workunit.client.0.trial128.stdout: "schedule": "scrub scheduled @ 2026-02-17T12:29:38.238 (2026-02-11T12:27:52.944)" 2026-02-17T12:29:38.186 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:38.186 INFO:tasks.workunit.client.0.trial128.stdout: "agent_state": {} 2026-02-17T12:29:38.266 INFO:tasks.workunit.client.0.trial128.stdout:( 2026-02-17T12:29:38.267 INFO:tasks.workunit.client.0.trial128.stdout:[query_epoch]=19 2026-02-17T12:29:38.267 INFO:tasks.workunit.client.0.trial128.stdout:[query_seq]=140 2026-02-17T12:29:38.267 INFO:tasks.workunit.client.0.trial128.stdout:[query_active]=false 2026-02-17T12:29:38.267 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule]='queued for scrub' 2026-02-17T12:29:38.267 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule_at]='0' 2026-02-17T12:29:38.267 INFO:tasks.workunit.client.0.trial128.stdout:[query_target_at]='0' 2026-02-17T12:29:38.267 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_duration]=1 2026-02-17T12:29:38.267 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_stamp]='2026-02-10T12:27:52.944114+0000' 2026-02-17T12:29:38.267 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_scrub]='17x90' 2026-02-17T12:29:38.267 INFO:tasks.workunit.client.0.trial128.stdout:[query_is_future]=false 2026-02-17T12:29:38.267 INFO:tasks.workunit.client.0.trial128.stdout:[query_vs_date]=false 2026-02-17T12:29:38.268 INFO:tasks.workunit.client.0.trial128.stdout:[query_scrub_seq]=null 2026-02-17T12:29:38.268 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_pg_state]='active+clean' 2026-02-17T12:29:38.268 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T12:29:38.268 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_last_duration]=1 2026-02-17T12:29:38.268 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T12:29:38.268 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule_at]='2026-02-17T12:29:38.238055+0000' 2026-02-17T12:29:38.268 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_is_future]=true 2026-02-17T12:29:38.268 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_vs_date]=true 2026-02-17T12:29:38.268 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_reported_epoch]=18 2026-02-17T12:29:38.268 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_seq]=136 2026-02-17T12:29:38.268 INFO:tasks.workunit.client.0.trial128.stdout:) 2026-02-17T12:29:38.278 INFO:tasks.workunit.client.0.trial128.stdout:--> loop: 4 ~ false / 140 / 136 / false / 2026-02-10T12:27:52.944114+0000 / queued for scrub %%% query_active 2026-02-17T12:29:38.278 INFO:tasks.workunit.client.0.trial128.stdout:key is query_active: negation:0 # expected: true # in actual: false 2026-02-17T12:29:38.926 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:29:38.936 INFO:tasks.workunit.client.0.trial128.stdout:from pg dump pg: { 2026-02-17T12:29:38.936 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_pg_state": "active+clean", 2026-02-17T12:29:38.936 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T12:29:38.936 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_last_duration": 1, 2026-02-17T12:29:38.936 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T12:29:38.936 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule_at": "2026-02-17T12:29:38.238055+0000", 2026-02-17T12:29:38.936 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_is_future": true, 2026-02-17T12:29:38.936 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_vs_date": true, 2026-02-17T12:29:38.936 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_reported_epoch": 18, 2026-02-17T12:29:38.937 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_seq": 136 2026-02-17T12:29:38.937 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:38.937 INFO:tasks.workunit.client.0.trial128.stdout:query output: 2026-02-17T12:29:39.005 INFO:tasks.workunit.client.0.trial128.stdout: "scrubber": { 2026-02-17T12:29:39.005 INFO:tasks.workunit.client.0.trial128.stdout: "active": false, 2026-02-17T12:29:39.005 INFO:tasks.workunit.client.0.trial128.stdout: "must_scrub": false, 2026-02-17T12:29:39.005 INFO:tasks.workunit.client.0.trial128.stdout: "must_deep_scrub": false, 2026-02-17T12:29:39.005 INFO:tasks.workunit.client.0.trial128.stdout: "must_repair": false, 2026-02-17T12:29:39.005 INFO:tasks.workunit.client.0.trial128.stdout: "need_auto": false, 2026-02-17T12:29:39.005 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reg_stamp": "2026-02-17T12:29:38.238055+0000", 2026-02-17T12:29:39.005 INFO:tasks.workunit.client.0.trial128.stdout: "schedule": "queued for scrub" 2026-02-17T12:29:39.006 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:39.006 INFO:tasks.workunit.client.0.trial128.stdout: "agent_state": {} 2026-02-17T12:29:39.086 INFO:tasks.workunit.client.0.trial128.stdout:( 2026-02-17T12:29:39.087 INFO:tasks.workunit.client.0.trial128.stdout:[query_epoch]=19 2026-02-17T12:29:39.087 INFO:tasks.workunit.client.0.trial128.stdout:[query_seq]=140 2026-02-17T12:29:39.087 INFO:tasks.workunit.client.0.trial128.stdout:[query_active]=false 2026-02-17T12:29:39.087 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule]='queued for scrub' 2026-02-17T12:29:39.087 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule_at]='0' 2026-02-17T12:29:39.087 INFO:tasks.workunit.client.0.trial128.stdout:[query_target_at]='0' 2026-02-17T12:29:39.087 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_duration]=1 2026-02-17T12:29:39.087 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_stamp]='2026-02-10T12:27:52.944114+0000' 2026-02-17T12:29:39.087 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_scrub]='17x90' 2026-02-17T12:29:39.087 INFO:tasks.workunit.client.0.trial128.stdout:[query_is_future]=false 2026-02-17T12:29:39.088 INFO:tasks.workunit.client.0.trial128.stdout:[query_vs_date]=false 2026-02-17T12:29:39.088 INFO:tasks.workunit.client.0.trial128.stdout:[query_scrub_seq]=null 2026-02-17T12:29:39.088 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_pg_state]='active+clean' 2026-02-17T12:29:39.088 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T12:29:39.088 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_last_duration]=1 2026-02-17T12:29:39.088 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T12:29:39.088 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule_at]='2026-02-17T12:29:38.238055+0000' 2026-02-17T12:29:39.088 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_is_future]=true 2026-02-17T12:29:39.088 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_vs_date]=true 2026-02-17T12:29:39.088 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_reported_epoch]=18 2026-02-17T12:29:39.088 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_seq]=136 2026-02-17T12:29:39.088 INFO:tasks.workunit.client.0.trial128.stdout:) 2026-02-17T12:29:39.098 INFO:tasks.workunit.client.0.trial128.stdout:--> loop: 5 ~ false / 140 / 136 / false / 2026-02-10T12:27:52.944114+0000 / queued for scrub %%% query_active 2026-02-17T12:29:39.099 INFO:tasks.workunit.client.0.trial128.stdout:key is query_active: negation:0 # expected: true # in actual: false 2026-02-17T12:29:39.741 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:29:39.750 INFO:tasks.workunit.client.0.trial128.stdout:from pg dump pg: { 2026-02-17T12:29:39.751 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_pg_state": "active+clean", 2026-02-17T12:29:39.751 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T12:29:39.751 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_last_duration": 1, 2026-02-17T12:29:39.751 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T12:29:39.751 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule_at": "2026-02-17T12:29:38.238055+0000", 2026-02-17T12:29:39.751 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_is_future": true, 2026-02-17T12:29:39.751 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_vs_date": true, 2026-02-17T12:29:39.751 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_reported_epoch": 19, 2026-02-17T12:29:39.751 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_seq": 139 2026-02-17T12:29:39.751 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:39.752 INFO:tasks.workunit.client.0.trial128.stdout:query output: 2026-02-17T12:29:39.825 INFO:tasks.workunit.client.0.trial128.stdout: "scrubber": { 2026-02-17T12:29:39.825 INFO:tasks.workunit.client.0.trial128.stdout: "active": true, 2026-02-17T12:29:39.825 INFO:tasks.workunit.client.0.trial128.stdout: "epoch_start": "15", 2026-02-17T12:29:39.825 INFO:tasks.workunit.client.0.trial128.stdout: "start": "1:00000000::::0", 2026-02-17T12:29:39.825 INFO:tasks.workunit.client.0.trial128.stdout: "end": "MIN", 2026-02-17T12:29:39.825 INFO:tasks.workunit.client.0.trial128.stdout: "max_end": "MIN", 2026-02-17T12:29:39.825 INFO:tasks.workunit.client.0.trial128.stdout: "subset_last_update": "0'0", 2026-02-17T12:29:39.825 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:29:39.825 INFO:tasks.workunit.client.0.trial128.stdout: "req_scrub": false, 2026-02-17T12:29:39.826 INFO:tasks.workunit.client.0.trial128.stdout: "auto_repair": false, 2026-02-17T12:29:39.826 INFO:tasks.workunit.client.0.trial128.stdout: "check_repair": false, 2026-02-17T12:29:39.826 INFO:tasks.workunit.client.0.trial128.stdout: "deep_scrub_on_error": false, 2026-02-17T12:29:39.826 INFO:tasks.workunit.client.0.trial128.stdout: "priority": 5, 2026-02-17T12:29:39.826 INFO:tasks.workunit.client.0.trial128.stdout: "shallow_errors": 0, 2026-02-17T12:29:39.826 INFO:tasks.workunit.client.0.trial128.stdout: "deep_errors": 0, 2026-02-17T12:29:39.826 INFO:tasks.workunit.client.0.trial128.stdout: "fixed": 0, 2026-02-17T12:29:39.826 INFO:tasks.workunit.client.0.trial128.stdout: "waiting_on_whom": [], 2026-02-17T12:29:39.826 INFO:tasks.workunit.client.0.trial128.stdout: "schedule": "scrubbing" 2026-02-17T12:29:39.826 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:39.826 INFO:tasks.workunit.client.0.trial128.stdout: "agent_state": {} 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:( 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[query_epoch]=19 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[query_seq]=144 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[query_active]=true 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule]='scrubbing' 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule_at]='0' 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[query_target_at]='0' 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_duration]=1 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_stamp]='2026-02-10T12:27:52.944114+0000' 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_scrub]='17x90' 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[query_is_future]=false 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[query_vs_date]=false 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[query_scrub_seq]=null 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_pg_state]='active+clean' 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_last_duration]=1 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule_at]='2026-02-17T12:29:38.238055+0000' 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_is_future]=true 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_vs_date]=true 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_reported_epoch]=19 2026-02-17T12:29:39.906 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_seq]=139 2026-02-17T12:29:39.907 INFO:tasks.workunit.client.0.trial128.stdout:) 2026-02-17T12:29:39.918 INFO:tasks.workunit.client.0.trial128.stdout:--> loop: 6 ~ true / 144 / 139 / false / 2026-02-10T12:27:52.944114+0000 / scrubbing %%% query_active 2026-02-17T12:29:39.918 INFO:tasks.workunit.client.0.trial128.stdout:key is query_active: negation:0 # expected: true # in actual: true 2026-02-17T12:29:39.918 INFO:tasks.workunit.client.0.trial128.stdout:WaitingActive - 'query_active' actual value (true) matches expected (true) (negation: 0) 2026-02-17T12:29:39.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-17T12:29:39.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:643: TEST_dump_scrub_schedule: cond_active_dmp=(['dmp_state_has_scrubbing']='true' ['query_active']='false') 2026-02-17T12:29:39.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:643: TEST_dump_scrub_schedule: declare -A cond_active_dmp 2026-02-17T12:29:39.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:644: TEST_dump_scrub_schedule: sched_data=() 2026-02-17T12:29:39.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:645: TEST_dump_scrub_schedule: wait_any_cond 1.0 10 2026-02-10T12:27:52.944114+0000 cond_active_dmp 'WaitingActive ' sched_data 2026-02-17T12:29:39.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.0 2026-02-17T12:29:39.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-17T12:29:39.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-10T12:27:52.944114+0000 2026-02-17T12:29:39.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=cond_active_dmp 2026-02-17T12:29:39.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sched_data 2026-02-17T12:29:39.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-17T12:29:39.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-17T12:29:39.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-17T12:29:39.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-17T12:29:39.920 INFO:tasks.workunit.client.0.trial128.stdout:waiting for any condition (WaitingActive ): pg:1.0 dt:2026-02-10T12:27:52.944114+0000 (10 retries) 2026-02-17T12:29:40.568 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:29:40.578 INFO:tasks.workunit.client.0.trial128.stdout:from pg dump pg: { 2026-02-17T12:29:40.578 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_pg_state": "active+clean", 2026-02-17T12:29:40.578 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T12:29:40.578 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_last_duration": 1, 2026-02-17T12:29:40.578 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T12:29:40.578 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule_at": "2026-02-17T12:29:38.238055+0000", 2026-02-17T12:29:40.578 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_is_future": false, 2026-02-17T12:29:40.578 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_vs_date": true, 2026-02-17T12:29:40.578 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_reported_epoch": 19, 2026-02-17T12:29:40.578 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_seq": 139 2026-02-17T12:29:40.579 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:40.579 INFO:tasks.workunit.client.0.trial128.stdout:query output: 2026-02-17T12:29:40.648 INFO:tasks.workunit.client.0.trial128.stdout: "scrubber": { 2026-02-17T12:29:40.648 INFO:tasks.workunit.client.0.trial128.stdout: "active": true, 2026-02-17T12:29:40.648 INFO:tasks.workunit.client.0.trial128.stdout: "epoch_start": "15", 2026-02-17T12:29:40.648 INFO:tasks.workunit.client.0.trial128.stdout: "start": "1:00000000::::0", 2026-02-17T12:29:40.648 INFO:tasks.workunit.client.0.trial128.stdout: "end": "MIN", 2026-02-17T12:29:40.648 INFO:tasks.workunit.client.0.trial128.stdout: "max_end": "MIN", 2026-02-17T12:29:40.649 INFO:tasks.workunit.client.0.trial128.stdout: "subset_last_update": "0'0", 2026-02-17T12:29:40.649 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:29:40.649 INFO:tasks.workunit.client.0.trial128.stdout: "req_scrub": false, 2026-02-17T12:29:40.649 INFO:tasks.workunit.client.0.trial128.stdout: "auto_repair": false, 2026-02-17T12:29:40.649 INFO:tasks.workunit.client.0.trial128.stdout: "check_repair": false, 2026-02-17T12:29:40.649 INFO:tasks.workunit.client.0.trial128.stdout: "deep_scrub_on_error": false, 2026-02-17T12:29:40.649 INFO:tasks.workunit.client.0.trial128.stdout: "priority": 5, 2026-02-17T12:29:40.649 INFO:tasks.workunit.client.0.trial128.stdout: "shallow_errors": 0, 2026-02-17T12:29:40.649 INFO:tasks.workunit.client.0.trial128.stdout: "deep_errors": 0, 2026-02-17T12:29:40.649 INFO:tasks.workunit.client.0.trial128.stdout: "fixed": 0, 2026-02-17T12:29:40.649 INFO:tasks.workunit.client.0.trial128.stdout: "waiting_on_whom": [], 2026-02-17T12:29:40.649 INFO:tasks.workunit.client.0.trial128.stdout: "schedule": "scrubbing" 2026-02-17T12:29:40.650 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:40.650 INFO:tasks.workunit.client.0.trial128.stdout: "agent_state": {} 2026-02-17T12:29:40.730 INFO:tasks.workunit.client.0.trial128.stdout:( 2026-02-17T12:29:40.731 INFO:tasks.workunit.client.0.trial128.stdout:[query_epoch]=19 2026-02-17T12:29:40.731 INFO:tasks.workunit.client.0.trial128.stdout:[query_seq]=144 2026-02-17T12:29:40.731 INFO:tasks.workunit.client.0.trial128.stdout:[query_active]=true 2026-02-17T12:29:40.731 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule]='scrubbing' 2026-02-17T12:29:40.731 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule_at]='0' 2026-02-17T12:29:40.731 INFO:tasks.workunit.client.0.trial128.stdout:[query_target_at]='0' 2026-02-17T12:29:40.731 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_duration]=1 2026-02-17T12:29:40.731 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_stamp]='2026-02-10T12:27:52.944114+0000' 2026-02-17T12:29:40.731 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_scrub]='17x90' 2026-02-17T12:29:40.731 INFO:tasks.workunit.client.0.trial128.stdout:[query_is_future]=false 2026-02-17T12:29:40.732 INFO:tasks.workunit.client.0.trial128.stdout:[query_vs_date]=false 2026-02-17T12:29:40.732 INFO:tasks.workunit.client.0.trial128.stdout:[query_scrub_seq]=null 2026-02-17T12:29:40.732 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_pg_state]='active+clean' 2026-02-17T12:29:40.732 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T12:29:40.732 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_last_duration]=1 2026-02-17T12:29:40.732 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T12:29:40.732 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule_at]='2026-02-17T12:29:38.238055+0000' 2026-02-17T12:29:40.732 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_is_future]=false 2026-02-17T12:29:40.732 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_vs_date]=true 2026-02-17T12:29:40.732 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_reported_epoch]=19 2026-02-17T12:29:40.732 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_seq]=139 2026-02-17T12:29:40.733 INFO:tasks.workunit.client.0.trial128.stdout:) 2026-02-17T12:29:40.742 INFO:tasks.workunit.client.0.trial128.stdout:--> loop: 1 ~ true / 144 / 139 / false / 2026-02-10T12:27:52.944114+0000 / scrubbing %%% dmp_state_has_scrubbing query_active 2026-02-17T12:29:40.742 INFO:tasks.workunit.client.0.trial128.stdout:key is dmp_state_has_scrubbing: negation:0 # expected: true # in actual: false 2026-02-17T12:29:40.742 INFO:tasks.workunit.client.0.trial128.stdout:key is query_active: negation:0 # expected: false # in actual: true 2026-02-17T12:29:41.385 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:29:41.394 INFO:tasks.workunit.client.0.trial128.stdout:from pg dump pg: { 2026-02-17T12:29:41.394 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_pg_state": "active+clean+scrubbing", 2026-02-17T12:29:41.394 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_state_has_scrubbing": true, 2026-02-17T12:29:41.394 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_last_duration": 1, 2026-02-17T12:29:41.394 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule": "scrubbing for 1s", 2026-02-17T12:29:41.395 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule_at": "0", 2026-02-17T12:29:41.395 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_is_future": false, 2026-02-17T12:29:41.395 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_vs_date": false, 2026-02-17T12:29:41.395 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_reported_epoch": 19, 2026-02-17T12:29:41.395 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_seq": 143 2026-02-17T12:29:41.395 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:41.395 INFO:tasks.workunit.client.0.trial128.stdout:query output: 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "scrubber": { 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "active": true, 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "epoch_start": "15", 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "start": "1:a0217245:::obj31:0", 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "end": "1:a0217245:::obj31:0", 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "max_end": "1:a0217245:::obj31:0", 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "subset_last_update": "17'90", 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "req_scrub": false, 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "auto_repair": false, 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "check_repair": false, 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "deep_scrub_on_error": false, 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "priority": 5, 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "shallow_errors": 0, 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "deep_errors": 0, 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "fixed": 0, 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "waiting_on_whom": [], 2026-02-17T12:29:41.465 INFO:tasks.workunit.client.0.trial128.stdout: "schedule": "scrubbing" 2026-02-17T12:29:41.466 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:41.466 INFO:tasks.workunit.client.0.trial128.stdout: "agent_state": {} 2026-02-17T12:29:41.546 INFO:tasks.workunit.client.0.trial128.stdout:( 2026-02-17T12:29:41.547 INFO:tasks.workunit.client.0.trial128.stdout:[query_epoch]=19 2026-02-17T12:29:41.547 INFO:tasks.workunit.client.0.trial128.stdout:[query_seq]=146 2026-02-17T12:29:41.547 INFO:tasks.workunit.client.0.trial128.stdout:[query_active]=true 2026-02-17T12:29:41.547 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule]='scrubbing' 2026-02-17T12:29:41.547 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule_at]='0' 2026-02-17T12:29:41.547 INFO:tasks.workunit.client.0.trial128.stdout:[query_target_at]='0' 2026-02-17T12:29:41.547 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_duration]=1 2026-02-17T12:29:41.547 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_stamp]='2026-02-10T12:27:52.944114+0000' 2026-02-17T12:29:41.548 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_scrub]='17x90' 2026-02-17T12:29:41.548 INFO:tasks.workunit.client.0.trial128.stdout:[query_is_future]=false 2026-02-17T12:29:41.548 INFO:tasks.workunit.client.0.trial128.stdout:[query_vs_date]=false 2026-02-17T12:29:41.548 INFO:tasks.workunit.client.0.trial128.stdout:[query_scrub_seq]=null 2026-02-17T12:29:41.548 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_pg_state]='active+clean+scrubbing' 2026-02-17T12:29:41.548 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_state_has_scrubbing]=true 2026-02-17T12:29:41.548 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_last_duration]=1 2026-02-17T12:29:41.548 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule]='scrubbing for 1s' 2026-02-17T12:29:41.548 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule_at]='0' 2026-02-17T12:29:41.548 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_is_future]=false 2026-02-17T12:29:41.548 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_vs_date]=false 2026-02-17T12:29:41.548 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_reported_epoch]=19 2026-02-17T12:29:41.548 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_seq]=143 2026-02-17T12:29:41.549 INFO:tasks.workunit.client.0.trial128.stdout:) 2026-02-17T12:29:41.558 INFO:tasks.workunit.client.0.trial128.stdout:--> loop: 2 ~ true / 146 / 143 / false / 2026-02-10T12:27:52.944114+0000 / scrubbing %%% dmp_state_has_scrubbing query_active 2026-02-17T12:29:41.559 INFO:tasks.workunit.client.0.trial128.stdout:key is dmp_state_has_scrubbing: negation:0 # expected: true # in actual: true 2026-02-17T12:29:41.559 INFO:tasks.workunit.client.0.trial128.stdout:WaitingActive - 'dmp_state_has_scrubbing' actual value (true) matches expected (true) (negation: 0) 2026-02-17T12:29:41.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-17T12:29:41.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:646: TEST_dump_scrub_schedule: sleep 4 2026-02-17T12:29:45.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:647: TEST_dump_scrub_schedule: perf_counters td/osd-scrub-test 3 2026-02-17T12:29:45.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-17T12:29:45.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-17T12:29:45.561 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-17T12:29:45.562 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-17T12:29:45.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:29:45.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-17T12:29:45.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:29:45.634 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:29:45.634 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:29:45.634 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:29:45.634 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:29:45.634 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:29:45.634 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:29:45.634 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.634 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:29:45.635 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:29:45.635 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:29:45.635 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:29:45.635 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:29:45.635 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.635 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.635 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.635 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.635 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:29:45.635 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:29:45.635 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.635 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.636 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.636 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.636 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:29:45.636 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:29:45.636 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:29:45.636 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:29:45.636 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:29:45.636 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:29:45.636 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:29:45.636 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.636 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.636 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.637 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.637 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:29:45.637 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:29:45.637 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:29:45.637 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:29:45.637 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.637 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.637 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.637 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.637 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:29:45.637 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.637 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.638 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:29:45.638 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:29:45.638 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:29:45.638 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:29:45.638 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:29:45.638 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:29:45.638 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.638 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:29:45.638 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:29:45.638 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:29:45.638 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:29:45.638 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:29:45.639 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.639 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.639 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.639 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.639 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:29:45.639 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:29:45.639 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.639 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.639 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.639 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.639 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:29:45.639 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:29:45.639 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:29:45.640 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:29:45.640 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:29:45.640 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:29:45.640 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:29:45.640 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.640 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.640 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.640 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.640 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:29:45.640 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:29:45.640 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:29:45.640 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:29:45.640 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.641 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.641 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.641 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.641 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:29:45.641 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.641 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.641 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:29:45.641 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:29:45.641 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:29:45.641 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:29:45.641 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:29:45.641 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:29:45.642 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.642 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:29:45.642 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:29:45.642 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:29:45.642 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:29:45.642 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:29:45.642 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.642 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.642 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.642 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.642 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:29:45.642 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:29:45.642 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.643 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.643 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.643 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.643 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:29:45.643 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:29:45.643 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:29:45.643 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:29:45.643 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:29:45.643 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:29:45.643 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:29:45.643 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.643 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.643 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.644 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.644 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:29:45.644 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:29:45.644 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:29:45.644 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:29:45.644 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.644 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.644 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.644 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.644 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:29:45.644 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.644 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.645 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:29:45.645 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:29:45.645 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:29:45.645 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:29:45.645 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:29:45.645 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:29:45.645 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.645 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:29:45.645 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:29:45.645 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:29:45.645 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:29:45.645 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:29:45.646 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.646 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.646 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.646 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.646 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:29:45.646 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:29:45.646 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.646 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.646 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.646 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.646 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:29:45.646 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:29:45.646 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:29:45.647 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:29:45.647 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:29:45.647 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:29:45.647 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:29:45.647 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.647 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.648 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.648 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.648 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:29:45.648 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:29:45.648 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:29:45.648 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:29:45.648 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.648 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.648 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.648 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.648 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:29:45.648 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.649 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.649 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:29:45.649 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:45.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:29:45.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-17T12:29:45.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:29:45.705 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:29:45.705 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:29:45.705 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:29:45.705 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:29:45.705 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:29:45.705 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:29:45.705 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.705 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:29:45.705 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:29:45.705 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 1, 2026-02-17T12:29:45.706 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 1, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 1, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 1, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0.019999656, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0.019999656 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 7, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 1, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 2 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:29:45.707 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 1, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 1, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 1, 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:29:45.708 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 1, 2026-02-17T12:29:45.709 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 4.007930339, 2026-02-17T12:29:45.709 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 4.007930339 2026-02-17T12:29:45.709 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.709 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:29:45.709 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:29:45.709 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.709 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.709 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.709 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.709 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:29:45.709 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 2, 2026-02-17T12:29:45.709 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:29:45.709 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:29:45.709 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:29:45.709 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 1, 2026-02-17T12:29:45.711 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:29:45.711 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 1, 2026-02-17T12:29:45.711 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.711 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.711 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.711 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:29:45.711 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:29:45.711 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:29:45.711 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:29:45.711 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.711 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.711 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.711 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.711 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 2 2026-02-17T12:29:45.711 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.712 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.712 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:29:45.712 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:45.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:29:45.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-17T12:29:45.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:29:45.778 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:29:45.778 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:29:45.779 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:29:45.779 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:29:45.779 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:29:45.779 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:29:45.779 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.779 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:29:45.779 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:29:45.779 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:29:45.779 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:29:45.779 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:29:45.779 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.780 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.780 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.780 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.780 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:29:45.780 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:29:45.780 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.780 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.780 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.780 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.780 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:29:45.781 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:29:45.781 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:29:45.781 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:29:45.781 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:29:45.781 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:29:45.781 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:29:45.781 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.781 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.781 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.781 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.781 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:29:45.782 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:29:45.782 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:29:45.782 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:29:45.782 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.782 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.782 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.782 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.782 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:29:45.782 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.782 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.782 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:29:45.783 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:29:45.783 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:29:45.783 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:29:45.783 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:29:45.783 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:29:45.783 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.783 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:29:45.783 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:29:45.783 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:29:45.783 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:29:45.783 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:29:45.783 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.784 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.784 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.784 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.784 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:29:45.784 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:29:45.784 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.784 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.784 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.784 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.784 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:29:45.784 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:29:45.784 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:29:45.784 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:29:45.785 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:29:45.785 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:29:45.785 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:29:45.785 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.785 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.785 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.785 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.785 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:29:45.785 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:29:45.785 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:29:45.785 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:29:45.785 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.785 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.786 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.786 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.786 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:29:45.786 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.786 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.786 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:29:45.786 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:29:45.786 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:29:45.786 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:29:45.786 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:29:45.786 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:29:45.786 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.786 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:29:45.787 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:29:45.787 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:29:45.787 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:29:45.787 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:29:45.787 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.787 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.787 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.787 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.787 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:29:45.787 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:29:45.787 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.787 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.787 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.788 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.788 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:29:45.788 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:29:45.788 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:29:45.788 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:29:45.788 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:29:45.788 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:29:45.788 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:29:45.788 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.788 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.788 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.788 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.788 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:29:45.789 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:29:45.789 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:29:45.789 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:29:45.789 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.789 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.789 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.789 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.789 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:29:45.789 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.789 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.790 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:29:45.790 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:29:45.790 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:29:45.790 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:29:45.790 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:29:45.790 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:29:45.790 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.790 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:29:45.790 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:29:45.790 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:29:45.790 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:29:45.790 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:29:45.791 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.791 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.791 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.791 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.791 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:29:45.791 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:29:45.791 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.791 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.791 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.791 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.791 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:29:45.791 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:29:45.791 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:29:45.792 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:29:45.792 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:29:45.792 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:29:45.792 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:29:45.793 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.793 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.793 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.793 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.793 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:29:45.793 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:29:45.793 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:29:45.793 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:29:45.793 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:29:45.793 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:29:45.793 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:29:45.794 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:29:45.794 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:29:45.794 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.794 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:29:45.794 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:29:45.794 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:29:45.794 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Teardown Test TEST_dump_scrub_schedule ------------------ 2026-02-17T12:29:45.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_dump_scrub_schedule ------------------' 2026-02-17T12:29:45.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T12:29:45.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:29:45.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:29:45.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:29:45.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:29:45.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:29:45.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:29:45.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:29:45.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:29:46.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:29:46.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:29:46.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:29:46.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:29:46.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:29:46.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:29:46.099 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:29:46.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:29:46.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:29:46.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:29:46.101 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:29:46.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:29:46.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:29:46.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:29:46.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:29:46.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:29:46.116 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:29:46.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:29:46.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:29:46.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:29:46.118 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Complete Test TEST_dump_scrub_schedule ------------------ 2026-02-17T12:29:46.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_dump_scrub_schedule ------------------' 2026-02-17T12:29:46.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T12:29:46.118 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Prepare Test TEST_interval_changes ------------------- 2026-02-17T12:29:46.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_interval_changes -------------------' 2026-02-17T12:29:46.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T12:29:46.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T12:29:46.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T12:29:46.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:29:46.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:29:46.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:29:46.119 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:29:46.119 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:29:46.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:29:46.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:29:46.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:29:46.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:29:46.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:29:46.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:29:46.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:29:46.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:29:46.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:29:46.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:29:46.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:29:46.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:29:46.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:29:46.126 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:29:46.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:29:46.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:29:46.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:29:46.130 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:29:46.130 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:29:46.130 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:29:46.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:29:46.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:29:46.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:29:46.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T12:29:46.132 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:29:46.132 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:29:46.133 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:29:46.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.508089 2026-02-17T12:29:46.134 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:29:46.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:29:46.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:29:46.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T12:29:46.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_interval_changes -----------------------' 2026-02-17T12:29:46.135 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Run Test TEST_interval_changes ----------------------- 2026-02-17T12:29:46.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_interval_changes td/osd-scrub-test 2026-02-17T12:29:46.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:149: TEST_interval_changes: local poolname=test 2026-02-17T12:29:46.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:150: TEST_interval_changes: local OSDS=2 2026-02-17T12:29:46.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:151: TEST_interval_changes: local objects=10 2026-02-17T12:29:46.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:153: TEST_interval_changes: expr 24 '*' 60 '*' 60 2026-02-17T12:29:46.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:153: TEST_interval_changes: local day=86400 2026-02-17T12:29:46.137 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:154: TEST_interval_changes: expr 86400 '*' 7 2026-02-17T12:29:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:154: TEST_interval_changes: local week=604800 2026-02-17T12:29:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:155: TEST_interval_changes: local min_interval=86400 2026-02-17T12:29:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:156: TEST_interval_changes: local max_interval=604800 2026-02-17T12:29:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:157: TEST_interval_changes: local WAIT_FOR_UPDATE=15 2026-02-17T12:29:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:159: TEST_interval_changes: TESTDATA=testdata.508089 2026-02-17T12:29:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:162: TEST_interval_changes: run_mon td/osd-scrub-test a --osd_pool_default_size=2 2026-02-17T12:29:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-test 2026-02-17T12:29:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:29:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:29:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:29:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-test/a 2026-02-17T12:29:46.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=2 2026-02-17T12:29:46.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:29:46.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:29:46.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:29:46.160 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:29:46.160 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:29:46.160 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:29:46.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:29:46.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-17T12:29:46.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:29:46.191 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:29:46.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:29:46.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:29:46.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:29:46.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:29:46.193 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:29:46.193 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:29:46.193 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:29:46.193 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:29:46.193 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:29:46.193 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:29:46.193 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-mon.a.asok 2026-02-17T12:29:46.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:29:46.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.508089/ceph-mon.a.asok config get fsid 2026-02-17T12:29:46.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:29:46.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:29:46.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:29:46.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:29:46.250 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:29:46.250 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:29:46.250 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:29:46.250 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:29:46.250 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:29:46.250 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:29:46.250 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:29:46.250 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-mon.a.asok 2026-02-17T12:29:46.251 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:29:46.251 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.508089/ceph-mon.a.asok config get mon_host 2026-02-17T12:29:46.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:163: TEST_interval_changes: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-17T12:29:46.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-test 2026-02-17T12:29:46.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:29:46.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:29:46.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:29:46.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-test/x 2026-02-17T12:29:46.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:29:46.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:29:46.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:29:46.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:29:46.402 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:29:46.402 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:29:46.402 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:29:46.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:29:46.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:29:46.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T12:29:46.420 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:164: TEST_interval_changes: expr 2 - 1 2026-02-17T12:29:46.421 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:164: TEST_interval_changes: seq 0 1 2026-02-17T12:29:46.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:164: TEST_interval_changes: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:29:46.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:166: TEST_interval_changes: run_osd td/osd-scrub-test 0 --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 2026-02-17T12:29:46.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:29:46.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:29:46.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:29:46.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:29:46.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-17T12:29:46.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:29:46.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:29:46.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:29:46.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:29:46.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-17T12:29:46.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-17T12:29:46.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:29:46.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:29:46.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:29:46.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:29:46.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:29:46.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:29:46.426 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:29:46.426 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:29:46.426 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:29:46.427 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:29:46.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:29:46.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:29:46.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:29:46.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:29:46.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:29:46.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:29:46.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:29:46.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:29:46.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:29:46.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:29:46.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:29:46.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0' 2026-02-17T12:29:46.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-17T12:29:46.428 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:29:46.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=416e83a5-765c-45ea-8c1c-5ac7b50ade2a 2026-02-17T12:29:46.429 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 416e83a5-765c-45ea-8c1c-5ac7b50ade2a 2026-02-17T12:29:46.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 416e83a5-765c-45ea-8c1c-5ac7b50ade2a' 2026-02-17T12:29:46.429 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:29:46.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA6X5RpEVIJGhAAjDK9mY+gFKPybUn+YIUxqA== 2026-02-17T12:29:46.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA6X5RpEVIJGhAAjDK9mY+gFKPybUn+YIUxqA=="}' 2026-02-17T12:29:46.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 416e83a5-765c-45ea-8c1c-5ac7b50ade2a -i td/osd-scrub-test/0/new.json 2026-02-17T12:29:46.529 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:29:46.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-17T12:29:46.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 --mkfs --key AQA6X5RpEVIJGhAAjDK9mY+gFKPybUn+YIUxqA== --osd-uuid 416e83a5-765c-45ea-8c1c-5ac7b50ade2a 2026-02-17T12:29:46.555 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:46.547+0000 7fc862c428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:46.556 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:46.547+0000 7fc862c428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:46.557 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:46.547+0000 7fc862c428c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:46.557 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:46.547+0000 7fc862c428c0 -1 bdev(0x557da663dc00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:29:46.558 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:46.547+0000 7fc862c428c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T12:29:49.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-17T12:29:49.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:29:49.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:29:49.181 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:29:49.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:29:49.369 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:29:49.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:29:49.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 2026-02-17T12:29:49.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:29:49.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:29:49.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:29:49.393 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:49.382+0000 7f4a5c9ae8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:49.394 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:49.386+0000 7f4a5c9ae8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:49.395 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:49.386+0000 7f4a5c9ae8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:49.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:29:49.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:29:49.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:29:49.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:29:49.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:29:49.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:29:49.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:29:49.521 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:29:49.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:29:49.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:29:49.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:29:50.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:29:50.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:29:50.676 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:29:50.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:29:50.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:29:50.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:29:50.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:29:50.896 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:50.886+0000 7f4a5c9ae8c0 -1 Falling back to public interface 2026-02-17T12:29:51.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:29:51.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:29:51.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:29:51.833 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:29:51.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:29:51.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:29:51.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:29:52.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:29:52.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:29:52.994 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:29:52.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:29:52.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:29:52.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:29:53.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:29:53.833 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:53.822+0000 7f4a5c9ae8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:29:54.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:29:54.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:29:54.149 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:29:54.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:29:54.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:29:54.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:29:54.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:29:55.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:29:55.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:29:55.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:29:55.300 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:29:55.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:29:55.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:29:55.452 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/207329312,v1:127.0.0.1:6803/207329312] [v2:127.0.0.1:6804/207329312,v1:127.0.0.1:6805/207329312] exists,up 416e83a5-765c-45ea-8c1c-5ac7b50ade2a 2026-02-17T12:29:55.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:29:55.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:29:55.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:29:55.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:164: TEST_interval_changes: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:29:55.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:166: TEST_interval_changes: run_osd td/osd-scrub-test 1 --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 2026-02-17T12:29:55.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:29:55.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:29:55.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:29:55.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:29:55.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-17T12:29:55.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:29:55.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:29:55.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:29:55.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:29:55.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-17T12:29:55.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-17T12:29:55.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:29:55.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:29:55.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:29:55.458 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:29:55.458 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:29:55.458 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:29:55.458 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:29:55.458 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:29:55.458 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:29:55.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:29:55.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:29:55.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:29:55.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:29:55.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:29:55.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:29:55.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:29:55.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:29:55.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:29:55.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:29:55.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:29:55.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:29:55.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0' 2026-02-17T12:29:55.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-17T12:29:55.460 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:29:55.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=575f8f61-0621-4fdc-87be-586f3453488a 2026-02-17T12:29:55.460 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 575f8f61-0621-4fdc-87be-586f3453488a 2026-02-17T12:29:55.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 575f8f61-0621-4fdc-87be-586f3453488a' 2026-02-17T12:29:55.461 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:29:55.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBDX5Rp7/ihGxAAjKuHZ+Qw2xYRKJOvfatYZQ== 2026-02-17T12:29:55.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBDX5Rp7/ihGxAAjKuHZ+Qw2xYRKJOvfatYZQ=="}' 2026-02-17T12:29:55.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 575f8f61-0621-4fdc-87be-586f3453488a -i td/osd-scrub-test/1/new.json 2026-02-17T12:29:55.625 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:29:55.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-17T12:29:55.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 --mkfs --key AQBDX5Rp7/ihGxAAjKuHZ+Qw2xYRKJOvfatYZQ== --osd-uuid 575f8f61-0621-4fdc-87be-586f3453488a 2026-02-17T12:29:55.652 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:55.642+0000 7fcbfec848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:55.653 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:55.642+0000 7fcbfec848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:55.654 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:55.646+0000 7fcbfec848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:55.655 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:55.646+0000 7fcbfec848c0 -1 bdev(0x55d3ac76bc00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:29:55.655 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:55.646+0000 7fcbfec848c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T12:29:57.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-17T12:29:57.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:29:57.912 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:29:57.913 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:29:57.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:29:58.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:29:58.110 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:29:58.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_scrub_min_interval=86400 --osd_scrub_max_interval=604800 --osd_scrub_interval_randomize_ratio=0 2026-02-17T12:29:58.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:29:58.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:29:58.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:29:58.133 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:58.122+0000 7f468d41e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:58.135 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:58.126+0000 7f468d41e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:58.136 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:58.126+0000 7f468d41e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:29:58.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:29:58.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:29:58.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:29:58.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:29:58.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:29:58.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:29:58.264 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:29:58.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:29:58.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:29:58.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:29:58.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:29:59.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:29:59.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:29:59.421 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:29:59.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:29:59.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:29:59.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:29:59.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:29:59.620 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:29:59.610+0000 7f468d41e8c0 -1 Falling back to public interface 2026-02-17T12:30:00.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:30:00.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:30:00.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:30:00.577 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:30:00.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:30:00.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:30:00.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:30:01.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:30:01.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:30:01.733 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:30:01.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:30:01.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:30:01.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:30:01.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:30:02.574 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:30:02.562+0000 7f468d41e8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:30:02.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:30:02.887 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:30:02.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:30:02.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:30:02.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:30:02.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:30:03.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:30:04.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:30:04.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:30:04.039 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:30:04.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:30:04.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:30:04.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:30:04.193 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3602760436,v1:127.0.0.1:6811/3602760436] [v2:127.0.0.1:6812/3602760436,v1:127.0.0.1:6813/3602760436] exists,up 575f8f61-0621-4fdc-87be-586f3453488a 2026-02-17T12:30:04.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:30:04.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:30:04.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:30:04.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:170: TEST_interval_changes: create_pool test 1 1 2026-02-17T12:30:04.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T12:30:04.394 INFO:tasks.workunit.client.0.trial128.stderr:pool 'test' created 2026-02-17T12:30:04.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:30:05.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:171: TEST_interval_changes: wait_for_clean 2026-02-17T12:30:05.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:30:05.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:30:05.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:30:05.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:30:05.407 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:30:05.407 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:30:05.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:30:05.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:30:05.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:30:05.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:30:05.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:30:05.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:30:05.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:30:05.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:30:05.512 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:30:05.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:30:05.665 INFO:tasks.workunit.client.0.trial128.stderr:1' 2026-02-17T12:30:05.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:30:05.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:30:05.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:30:05.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836492 2026-02-17T12:30:05.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836492 2026-02-17T12:30:05.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492' 2026-02-17T12:30:05.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:30:05.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:30:05.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672964 2026-02-17T12:30:05.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672964 2026-02-17T12:30:05.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-42949672964' 2026-02-17T12:30:05.811 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:30:05.812 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836492 2026-02-17T12:30:05.812 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:30:05.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:30:05.813 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836492 2026-02-17T12:30:05.813 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:30:05.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836492 2026-02-17T12:30:05.815 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836492 2026-02-17T12:30:05.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-17T12:30:05.815 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:30:05.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836492 2026-02-17T12:30:05.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:30:06.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:30:06.969 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:30:07.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836492 2026-02-17T12:30:07.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:30:07.119 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672964 2026-02-17T12:30:07.119 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:30:07.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:30:07.121 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672964 2026-02-17T12:30:07.121 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:30:07.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672964 2026-02-17T12:30:07.123 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672964 2026-02-17T12:30:07.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-17T12:30:07.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:30:07.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-17T12:30:07.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:30:07.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:30:07.281 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:30:07.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:30:07.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:30:07.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:30:07.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:30:07.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:30:07.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:30:07.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:30:07.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:30:07.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:30:07.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:30:07.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:30:07.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:30:07.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:30:07.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:30:07.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:30:07.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:172: TEST_interval_changes: ceph osd dump 2026-02-17T12:30:07.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:172: TEST_interval_changes: grep '^pool.*['\'']test['\'']' 2026-02-17T12:30:07.809 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:172: TEST_interval_changes: awk '{ print $2 }' 2026-02-17T12:30:07.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:172: TEST_interval_changes: local poolid=1 2026-02-17T12:30:07.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:174: TEST_interval_changes: dd if=/dev/urandom of=testdata.508089 bs=1032 count=1 2026-02-17T12:30:07.963 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records in 2026-02-17T12:30:07.963 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records out 2026-02-17T12:30:07.963 INFO:tasks.workunit.client.0.trial128.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.5095e-05 s, 29.4 MB/s 2026-02-17T12:30:07.963 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: seq 1 10 2026-02-17T12:30:07.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T12:30:07.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj1 testdata.508089 2026-02-17T12:30:07.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T12:30:07.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj2 testdata.508089 2026-02-17T12:30:08.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T12:30:08.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj3 testdata.508089 2026-02-17T12:30:08.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T12:30:08.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj4 testdata.508089 2026-02-17T12:30:08.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T12:30:08.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj5 testdata.508089 2026-02-17T12:30:08.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T12:30:08.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj6 testdata.508089 2026-02-17T12:30:08.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T12:30:08.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj7 testdata.508089 2026-02-17T12:30:08.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T12:30:08.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj8 testdata.508089 2026-02-17T12:30:08.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T12:30:08.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj9 testdata.508089 2026-02-17T12:30:08.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:175: TEST_interval_changes: for i in `seq 1 $objects` 2026-02-17T12:30:08.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:177: TEST_interval_changes: rados -p test put obj10 testdata.508089 2026-02-17T12:30:08.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:179: TEST_interval_changes: rm -f testdata.508089 2026-02-17T12:30:08.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:181: TEST_interval_changes: get_primary test obj1 2026-02-17T12:30:08.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T12:30:08.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T12:30:08.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T12:30:08.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:30:08.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:181: TEST_interval_changes: local primary=1 2026-02-17T12:30:08.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:184: TEST_interval_changes: check_dump_scrubs 1 '1 day' '1 week' 2026-02-17T12:30:08.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: local primary=1 2026-02-17T12:30:08.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:136: check_dump_scrubs: local 'sched_time_check=1 day' 2026-02-17T12:30:08.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: local 'deadline_check=1 week' 2026-02-17T12:30:08.343 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: get_asok_path osd.1 2026-02-17T12:30:08.343 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:30:08.344 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:30:08.344 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:30:08.344 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:30:08.344 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:30:08.344 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-osd.1.asok 2026-02-17T12:30:08.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: CEPH_ARGS= 2026-02-17T12:30:08.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: ceph --admin-daemon /tmp/ceph-asok.508089/ceph-osd.1.asok dump_scrubs 2026-02-17T12:30:08.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: DS='[ 2026-02-17T12:30:08.403 INFO:tasks.workunit.client.0.trial128.stderr: { 2026-02-17T12:30:08.404 INFO:tasks.workunit.client.0.trial128.stderr: "pgid": "1.0", 2026-02-17T12:30:08.404 INFO:tasks.workunit.client.0.trial128.stderr: "sched_time": "2026-02-18T12:30:04.389162+0000", 2026-02-17T12:30:08.404 INFO:tasks.workunit.client.0.trial128.stderr: "orig_sched_time": "2026-02-18T12:30:04.389162+0000", 2026-02-17T12:30:08.404 INFO:tasks.workunit.client.0.trial128.stderr: "deadline": "2026-02-24T12:30:04.389162+0000", 2026-02-17T12:30:08.404 INFO:tasks.workunit.client.0.trial128.stderr: "forced": false 2026-02-17T12:30:08.404 INFO:tasks.workunit.client.0.trial128.stderr: } 2026-02-17T12:30:08.404 INFO:tasks.workunit.client.0.trial128.stderr:]' 2026-02-17T12:30:08.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-18T12:30:04.389162+0000",' '"orig_sched_time":' '"2026-02-18T12:30:04.389162+0000",' '"deadline":' '"2026-02-24T12:30:04.389162+0000",' '"forced":' false '}' ']' 2026-02-17T12:30:08.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: jq '.[0].sched_time' 2026-02-17T12:30:08.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: eval 'SCHED_TIME="2026-02-18T12:30:04.389162+0000"' 2026-02-17T12:30:08.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: SCHED_TIME=2026-02-18T12:30:04.389162+0000 2026-02-17T12:30:08.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: echo 2026-02-18T12:30:04.389162+0000 2026-02-17T12:30:08.415 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T12:30:08.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: date +%Y-%m-%d -d 'now + 1 day' 2026-02-17T12:30:08.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: test 2026-02-18 = 2026-02-18 2026-02-17T12:30:08.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-18T12:30:04.389162+0000",' '"orig_sched_time":' '"2026-02-18T12:30:04.389162+0000",' '"deadline":' '"2026-02-24T12:30:04.389162+0000",' '"forced":' false '}' ']' 2026-02-17T12:30:08.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: jq '.[0].deadline' 2026-02-17T12:30:08.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: eval 'DEADLINE="2026-02-24T12:30:04.389162+0000"' 2026-02-17T12:30:08.430 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: DEADLINE=2026-02-24T12:30:04.389162+0000 2026-02-17T12:30:08.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: echo 2026-02-24T12:30:04.389162+0000 2026-02-17T12:30:08.431 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T12:30:08.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: date +%Y-%m-%d -d 'now + 1 week' 2026-02-17T12:30:08.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: test 2026-02-24 = 2026-02-24 2026-02-17T12:30:08.434 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:187: TEST_interval_changes: get_asok_path osd.1 2026-02-17T12:30:08.434 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:30:08.435 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:30:08.435 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:30:08.435 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:30:08.435 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:30:08.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-osd.1.asok 2026-02-17T12:30:08.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:187: TEST_interval_changes: expr 86400 '*' 2 2026-02-17T12:30:08.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:187: TEST_interval_changes: CEPH_ARGS= 2026-02-17T12:30:08.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:187: TEST_interval_changes: ceph --admin-daemon /tmp/ceph-asok.508089/ceph-osd.1.asok config set osd_scrub_min_interval 172800 2026-02-17T12:30:08.489 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:30:08.489 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_min_interval = '' " 2026-02-17T12:30:08.489 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:30:08.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:188: TEST_interval_changes: sleep 15 2026-02-17T12:30:23.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:189: TEST_interval_changes: check_dump_scrubs 1 '2 days' '1 week' 2026-02-17T12:30:23.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: local primary=1 2026-02-17T12:30:23.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:136: check_dump_scrubs: local 'sched_time_check=2 days' 2026-02-17T12:30:23.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: local 'deadline_check=1 week' 2026-02-17T12:30:23.499 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: get_asok_path osd.1 2026-02-17T12:30:23.499 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:30:23.499 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:30:23.499 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:30:23.499 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:30:23.500 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:30:23.500 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-osd.1.asok 2026-02-17T12:30:23.500 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: CEPH_ARGS= 2026-02-17T12:30:23.500 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: ceph --admin-daemon /tmp/ceph-asok.508089/ceph-osd.1.asok dump_scrubs 2026-02-17T12:30:23.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: DS='[ 2026-02-17T12:30:23.557 INFO:tasks.workunit.client.0.trial128.stderr: { 2026-02-17T12:30:23.558 INFO:tasks.workunit.client.0.trial128.stderr: "pgid": "1.0", 2026-02-17T12:30:23.558 INFO:tasks.workunit.client.0.trial128.stderr: "sched_time": "2026-02-19T12:30:04.389162+0000", 2026-02-17T12:30:23.558 INFO:tasks.workunit.client.0.trial128.stderr: "orig_sched_time": "2026-02-19T12:30:04.389162+0000", 2026-02-17T12:30:23.558 INFO:tasks.workunit.client.0.trial128.stderr: "deadline": "2026-02-24T12:30:04.389162+0000", 2026-02-17T12:30:23.558 INFO:tasks.workunit.client.0.trial128.stderr: "forced": false 2026-02-17T12:30:23.558 INFO:tasks.workunit.client.0.trial128.stderr: } 2026-02-17T12:30:23.558 INFO:tasks.workunit.client.0.trial128.stderr:]' 2026-02-17T12:30:23.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-19T12:30:04.389162+0000",' '"orig_sched_time":' '"2026-02-19T12:30:04.389162+0000",' '"deadline":' '"2026-02-24T12:30:04.389162+0000",' '"forced":' false '}' ']' 2026-02-17T12:30:23.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: jq '.[0].sched_time' 2026-02-17T12:30:23.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: eval 'SCHED_TIME="2026-02-19T12:30:04.389162+0000"' 2026-02-17T12:30:23.567 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: SCHED_TIME=2026-02-19T12:30:04.389162+0000 2026-02-17T12:30:23.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: echo 2026-02-19T12:30:04.389162+0000 2026-02-17T12:30:23.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T12:30:23.570 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: date +%Y-%m-%d -d 'now + 2 days' 2026-02-17T12:30:23.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: test 2026-02-19 = 2026-02-19 2026-02-17T12:30:23.572 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-19T12:30:04.389162+0000",' '"orig_sched_time":' '"2026-02-19T12:30:04.389162+0000",' '"deadline":' '"2026-02-24T12:30:04.389162+0000",' '"forced":' false '}' ']' 2026-02-17T12:30:23.572 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: jq '.[0].deadline' 2026-02-17T12:30:23.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: eval 'DEADLINE="2026-02-24T12:30:04.389162+0000"' 2026-02-17T12:30:23.582 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: DEADLINE=2026-02-24T12:30:04.389162+0000 2026-02-17T12:30:23.583 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: echo 2026-02-24T12:30:04.389162+0000 2026-02-17T12:30:23.583 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T12:30:23.585 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: date +%Y-%m-%d -d 'now + 1 week' 2026-02-17T12:30:23.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: test 2026-02-24 = 2026-02-24 2026-02-17T12:30:23.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:192: TEST_interval_changes: get_asok_path osd.1 2026-02-17T12:30:23.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:30:23.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:30:23.587 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:30:23.587 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:30:23.587 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:30:23.587 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-osd.1.asok 2026-02-17T12:30:23.588 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:192: TEST_interval_changes: expr 604800 '*' 2 2026-02-17T12:30:23.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:192: TEST_interval_changes: CEPH_ARGS= 2026-02-17T12:30:23.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:192: TEST_interval_changes: ceph --admin-daemon /tmp/ceph-asok.508089/ceph-osd.1.asok config set osd_scrub_max_interval 1209600 2026-02-17T12:30:23.642 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:30:23.642 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_max_interval = '' " 2026-02-17T12:30:23.643 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:30:23.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:193: TEST_interval_changes: sleep 15 2026-02-17T12:30:38.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:194: TEST_interval_changes: check_dump_scrubs 1 '2 days' '2 week' 2026-02-17T12:30:38.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: local primary=1 2026-02-17T12:30:38.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:136: check_dump_scrubs: local 'sched_time_check=2 days' 2026-02-17T12:30:38.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: local 'deadline_check=2 week' 2026-02-17T12:30:38.650 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: get_asok_path osd.1 2026-02-17T12:30:38.651 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:30:38.651 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:30:38.651 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:30:38.651 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:30:38.652 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:30:38.652 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-osd.1.asok 2026-02-17T12:30:38.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: CEPH_ARGS= 2026-02-17T12:30:38.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: ceph --admin-daemon /tmp/ceph-asok.508089/ceph-osd.1.asok dump_scrubs 2026-02-17T12:30:38.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: DS='[ 2026-02-17T12:30:38.711 INFO:tasks.workunit.client.0.trial128.stderr: { 2026-02-17T12:30:38.711 INFO:tasks.workunit.client.0.trial128.stderr: "pgid": "1.0", 2026-02-17T12:30:38.711 INFO:tasks.workunit.client.0.trial128.stderr: "sched_time": "2026-02-19T12:30:04.389162+0000", 2026-02-17T12:30:38.711 INFO:tasks.workunit.client.0.trial128.stderr: "orig_sched_time": "2026-02-19T12:30:04.389162+0000", 2026-02-17T12:30:38.711 INFO:tasks.workunit.client.0.trial128.stderr: "deadline": "2026-03-03T12:30:04.389162+0000", 2026-02-17T12:30:38.711 INFO:tasks.workunit.client.0.trial128.stderr: "forced": false 2026-02-17T12:30:38.711 INFO:tasks.workunit.client.0.trial128.stderr: } 2026-02-17T12:30:38.712 INFO:tasks.workunit.client.0.trial128.stderr:]' 2026-02-17T12:30:38.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-19T12:30:04.389162+0000",' '"orig_sched_time":' '"2026-02-19T12:30:04.389162+0000",' '"deadline":' '"2026-03-03T12:30:04.389162+0000",' '"forced":' false '}' ']' 2026-02-17T12:30:38.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: jq '.[0].sched_time' 2026-02-17T12:30:38.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: eval 'SCHED_TIME="2026-02-19T12:30:04.389162+0000"' 2026-02-17T12:30:38.722 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: SCHED_TIME=2026-02-19T12:30:04.389162+0000 2026-02-17T12:30:38.722 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: echo 2026-02-19T12:30:04.389162+0000 2026-02-17T12:30:38.723 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T12:30:38.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: date +%Y-%m-%d -d 'now + 2 days' 2026-02-17T12:30:38.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: test 2026-02-19 = 2026-02-19 2026-02-17T12:30:38.726 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-19T12:30:04.389162+0000",' '"orig_sched_time":' '"2026-02-19T12:30:04.389162+0000",' '"deadline":' '"2026-03-03T12:30:04.389162+0000",' '"forced":' false '}' ']' 2026-02-17T12:30:38.726 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: jq '.[0].deadline' 2026-02-17T12:30:38.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: eval 'DEADLINE="2026-03-03T12:30:04.389162+0000"' 2026-02-17T12:30:38.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: DEADLINE=2026-03-03T12:30:04.389162+0000 2026-02-17T12:30:38.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: echo 2026-03-03T12:30:04.389162+0000 2026-02-17T12:30:38.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T12:30:38.738 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: date +%Y-%m-%d -d 'now + 2 week' 2026-02-17T12:30:38.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: test 2026-03-03 = 2026-03-03 2026-02-17T12:30:38.740 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:197: TEST_interval_changes: expr 86400 '*' 3 2026-02-17T12:30:38.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:197: TEST_interval_changes: ceph osd pool set test scrub_min_interval 259200 2026-02-17T12:30:38.937 INFO:tasks.workunit.client.0.trial128.stderr:set pool 1 scrub_min_interval to 259200 2026-02-17T12:30:38.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:198: TEST_interval_changes: sleep 15 2026-02-17T12:30:53.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:199: TEST_interval_changes: check_dump_scrubs 1 '3 days' '2 week' 2026-02-17T12:30:53.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: local primary=1 2026-02-17T12:30:53.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:136: check_dump_scrubs: local 'sched_time_check=3 days' 2026-02-17T12:30:53.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: local 'deadline_check=2 week' 2026-02-17T12:30:53.952 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: get_asok_path osd.1 2026-02-17T12:30:53.952 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:30:53.952 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:30:53.953 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:30:53.953 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:30:53.953 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:30:53.954 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-osd.1.asok 2026-02-17T12:30:53.954 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: CEPH_ARGS= 2026-02-17T12:30:53.954 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: ceph --admin-daemon /tmp/ceph-asok.508089/ceph-osd.1.asok dump_scrubs 2026-02-17T12:30:54.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: DS='[ 2026-02-17T12:30:54.013 INFO:tasks.workunit.client.0.trial128.stderr: { 2026-02-17T12:30:54.013 INFO:tasks.workunit.client.0.trial128.stderr: "pgid": "1.0", 2026-02-17T12:30:54.013 INFO:tasks.workunit.client.0.trial128.stderr: "sched_time": "2026-02-20T12:30:04.389162+0000", 2026-02-17T12:30:54.014 INFO:tasks.workunit.client.0.trial128.stderr: "orig_sched_time": "2026-02-20T12:30:04.389162+0000", 2026-02-17T12:30:54.014 INFO:tasks.workunit.client.0.trial128.stderr: "deadline": "2026-03-03T12:30:04.389162+0000", 2026-02-17T12:30:54.014 INFO:tasks.workunit.client.0.trial128.stderr: "forced": false 2026-02-17T12:30:54.014 INFO:tasks.workunit.client.0.trial128.stderr: } 2026-02-17T12:30:54.014 INFO:tasks.workunit.client.0.trial128.stderr:]' 2026-02-17T12:30:54.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-20T12:30:04.389162+0000",' '"orig_sched_time":' '"2026-02-20T12:30:04.389162+0000",' '"deadline":' '"2026-03-03T12:30:04.389162+0000",' '"forced":' false '}' ']' 2026-02-17T12:30:54.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: jq '.[0].sched_time' 2026-02-17T12:30:54.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: eval 'SCHED_TIME="2026-02-20T12:30:04.389162+0000"' 2026-02-17T12:30:54.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: SCHED_TIME=2026-02-20T12:30:04.389162+0000 2026-02-17T12:30:54.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: echo 2026-02-20T12:30:04.389162+0000 2026-02-17T12:30:54.025 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T12:30:54.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: date +%Y-%m-%d -d 'now + 3 days' 2026-02-17T12:30:54.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: test 2026-02-20 = 2026-02-20 2026-02-17T12:30:54.027 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-20T12:30:04.389162+0000",' '"orig_sched_time":' '"2026-02-20T12:30:04.389162+0000",' '"deadline":' '"2026-03-03T12:30:04.389162+0000",' '"forced":' false '}' ']' 2026-02-17T12:30:54.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: jq '.[0].deadline' 2026-02-17T12:30:54.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: eval 'DEADLINE="2026-03-03T12:30:04.389162+0000"' 2026-02-17T12:30:54.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: DEADLINE=2026-03-03T12:30:04.389162+0000 2026-02-17T12:30:54.039 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: echo 2026-03-03T12:30:04.389162+0000 2026-02-17T12:30:54.039 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T12:30:54.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: date +%Y-%m-%d -d 'now + 2 week' 2026-02-17T12:30:54.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: test 2026-03-03 = 2026-03-03 2026-02-17T12:30:54.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:202: TEST_interval_changes: expr 604800 '*' 3 2026-02-17T12:30:54.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:202: TEST_interval_changes: ceph osd pool set test scrub_max_interval 1814400 2026-02-17T12:30:54.232 INFO:tasks.workunit.client.0.trial128.stderr:set pool 1 scrub_max_interval to 1814400 2026-02-17T12:30:54.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:203: TEST_interval_changes: sleep 15 2026-02-17T12:31:09.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:204: TEST_interval_changes: check_dump_scrubs 1 '3 days' '3 week' 2026-02-17T12:31:09.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:135: check_dump_scrubs: local primary=1 2026-02-17T12:31:09.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:136: check_dump_scrubs: local 'sched_time_check=3 days' 2026-02-17T12:31:09.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:137: check_dump_scrubs: local 'deadline_check=3 week' 2026-02-17T12:31:09.246 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: get_asok_path osd.1 2026-02-17T12:31:09.246 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-17T12:31:09.246 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-17T12:31:09.247 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:31:09.247 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:31:09.247 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:31:09.247 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-osd.1.asok 2026-02-17T12:31:09.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: CEPH_ARGS= 2026-02-17T12:31:09.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: ceph --admin-daemon /tmp/ceph-asok.508089/ceph-osd.1.asok dump_scrubs 2026-02-17T12:31:09.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:139: check_dump_scrubs: DS='[ 2026-02-17T12:31:09.305 INFO:tasks.workunit.client.0.trial128.stderr: { 2026-02-17T12:31:09.305 INFO:tasks.workunit.client.0.trial128.stderr: "pgid": "1.0", 2026-02-17T12:31:09.305 INFO:tasks.workunit.client.0.trial128.stderr: "sched_time": "2026-02-20T12:30:04.389162+0000", 2026-02-17T12:31:09.305 INFO:tasks.workunit.client.0.trial128.stderr: "orig_sched_time": "2026-02-20T12:30:04.389162+0000", 2026-02-17T12:31:09.305 INFO:tasks.workunit.client.0.trial128.stderr: "deadline": "2026-03-10T12:30:04.389162+0000", 2026-02-17T12:31:09.305 INFO:tasks.workunit.client.0.trial128.stderr: "forced": false 2026-02-17T12:31:09.306 INFO:tasks.workunit.client.0.trial128.stderr: } 2026-02-17T12:31:09.306 INFO:tasks.workunit.client.0.trial128.stderr:]' 2026-02-17T12:31:09.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-20T12:30:04.389162+0000",' '"orig_sched_time":' '"2026-02-20T12:30:04.389162+0000",' '"deadline":' '"2026-03-10T12:30:04.389162+0000",' '"forced":' false '}' ']' 2026-02-17T12:31:09.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: jq '.[0].sched_time' 2026-02-17T12:31:09.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: eval 'SCHED_TIME="2026-02-20T12:30:04.389162+0000"' 2026-02-17T12:31:09.315 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:141: check_dump_scrubs: SCHED_TIME=2026-02-20T12:30:04.389162+0000 2026-02-17T12:31:09.316 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: echo 2026-02-20T12:30:04.389162+0000 2026-02-17T12:31:09.316 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T12:31:09.318 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: date +%Y-%m-%d -d 'now + 3 days' 2026-02-17T12:31:09.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:142: check_dump_scrubs: test 2026-02-20 = 2026-02-20 2026-02-17T12:31:09.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: echo '[' '{' '"pgid":' '"1.0",' '"sched_time":' '"2026-02-20T12:30:04.389162+0000",' '"orig_sched_time":' '"2026-02-20T12:30:04.389162+0000",' '"deadline":' '"2026-03-10T12:30:04.389162+0000",' '"forced":' false '}' ']' 2026-02-17T12:31:09.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: jq '.[0].deadline' 2026-02-17T12:31:09.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: eval 'DEADLINE="2026-03-10T12:30:04.389162+0000"' 2026-02-17T12:31:09.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:144: check_dump_scrubs: DEADLINE=2026-03-10T12:30:04.389162+0000 2026-02-17T12:31:09.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: echo 2026-03-10T12:30:04.389162+0000 2026-02-17T12:31:09.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: sed 's/\([0-9]*-[0-9]*-[0-9]*\).*/\1/' 2026-02-17T12:31:09.333 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: date +%Y-%m-%d -d 'now + 3 week' 2026-02-17T12:31:09.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:145: check_dump_scrubs: test 2026-03-10 = 2026-03-10 2026-02-17T12:31:09.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:205: TEST_interval_changes: perf_counters td/osd-scrub-test 2 2026-02-17T12:31:09.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-17T12:31:09.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=2 2026-02-17T12:31:09.335 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 2 - 1 2026-02-17T12:31:09.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 1 2026-02-17T12:31:09.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:31:09.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-17T12:31:09.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:31:09.410 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:31:09.410 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:31:09.410 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:31:09.411 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:31:09.411 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:31:09.411 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:31:09.411 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.411 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:31:09.411 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:31:09.411 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:31:09.411 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:31:09.411 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:31:09.411 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.411 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.412 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.412 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.412 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:31:09.412 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:31:09.412 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.412 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.412 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.412 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.412 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:31:09.412 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:31:09.413 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:31:09.413 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:31:09.413 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:31:09.413 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:31:09.413 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:31:09.413 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.413 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.413 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.413 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.413 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:31:09.413 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:31:09.414 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:31:09.414 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:31:09.414 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.414 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.414 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.414 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.414 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:31:09.414 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:31:09.414 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:31:09.414 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:31:09.414 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:31:09.415 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:31:09.415 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:31:09.415 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:31:09.415 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:31:09.415 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.415 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:31:09.415 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:31:09.415 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:31:09.415 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:31:09.415 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:31:09.416 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.416 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.416 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.416 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.416 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:31:09.416 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:31:09.416 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.416 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.416 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.416 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.416 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:31:09.417 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:31:09.417 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:31:09.417 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:31:09.417 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:31:09.417 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:31:09.417 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:31:09.417 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.417 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.417 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.417 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.417 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:31:09.418 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:31:09.418 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:31:09.418 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:31:09.418 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.418 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.418 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.418 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.418 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:31:09.418 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:31:09.418 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:31:09.418 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:31:09.419 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:31:09.419 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:31:09.419 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:31:09.419 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:31:09.419 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:31:09.419 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.419 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:31:09.419 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:31:09.419 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:31:09.419 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:31:09.419 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:31:09.419 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.420 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.420 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.420 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.420 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:31:09.420 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:31:09.420 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.420 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.420 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.420 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.420 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:31:09.421 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:31:09.421 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:31:09.421 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:31:09.421 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:31:09.421 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:31:09.421 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:31:09.421 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.421 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.421 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.421 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.421 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:31:09.422 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:31:09.422 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:31:09.422 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:31:09.422 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.422 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.422 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.422 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.422 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:31:09.422 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:31:09.422 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:31:09.422 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:31:09.422 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:31:09.423 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:31:09.423 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:31:09.423 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:31:09.423 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:31:09.423 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.423 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:31:09.423 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:31:09.423 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:31:09.423 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:31:09.423 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:31:09.423 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.424 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.424 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.424 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.424 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:31:09.424 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:31:09.424 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.424 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.424 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.424 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.424 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:31:09.424 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:31:09.424 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:31:09.424 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:31:09.425 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:31:09.425 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:31:09.425 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:31:09.425 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.425 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.426 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.426 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.426 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:31:09.426 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:31:09.426 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:31:09.426 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:31:09.426 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.426 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.426 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.426 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.427 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:31:09.427 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:31:09.427 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:31:09.427 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:31:09.427 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:31:09.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:31:09.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-17T12:31:09.428 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:31:09.481 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:31:09.482 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:31:09.482 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:31:09.482 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:31:09.482 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:31:09.482 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:31:09.482 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.482 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:31:09.482 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:31:09.482 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:31:09.482 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:31:09.483 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:31:09.483 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.483 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.483 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.483 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.483 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:31:09.483 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:31:09.483 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.483 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.483 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.483 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.483 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:31:09.483 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:31:09.484 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:31:09.484 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:31:09.484 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:31:09.484 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:31:09.484 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:31:09.484 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.484 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.484 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.484 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.484 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:31:09.485 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:31:09.485 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:31:09.485 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:31:09.485 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.485 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.485 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.485 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.485 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:31:09.485 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:31:09.485 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:31:09.485 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:31:09.486 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:31:09.486 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:31:09.486 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:31:09.486 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:31:09.486 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:31:09.486 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.486 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:31:09.486 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:31:09.487 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:31:09.487 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:31:09.487 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:31:09.487 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.487 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.487 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.487 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.487 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:31:09.487 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:31:09.487 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.487 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.487 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.487 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.488 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:31:09.488 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:31:09.488 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:31:09.488 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:31:09.488 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:31:09.488 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:31:09.488 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:31:09.488 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.488 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.488 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.488 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.488 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:31:09.488 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:31:09.489 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:31:09.489 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:31:09.489 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.489 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.489 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.489 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.489 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:31:09.489 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:31:09.489 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:31:09.489 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:31:09.489 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:31:09.489 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:31:09.490 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:31:09.490 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:31:09.490 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:31:09.490 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.490 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:31:09.490 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:31:09.490 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:31:09.490 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:31:09.490 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:31:09.490 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.491 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.491 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.491 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.491 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:31:09.491 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:31:09.491 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.491 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.491 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.491 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.491 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:31:09.492 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:31:09.492 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:31:09.492 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:31:09.492 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:31:09.492 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:31:09.492 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:31:09.492 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.492 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.492 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.492 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.492 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:31:09.493 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:31:09.493 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:31:09.493 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:31:09.493 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.493 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.493 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.493 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.493 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:31:09.493 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:31:09.493 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:31:09.493 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:31:09.494 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:31:09.494 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:31:09.494 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:31:09.494 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:31:09.494 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:31:09.494 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.494 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:31:09.494 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:31:09.494 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:31:09.494 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:31:09.494 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:31:09.495 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.495 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.495 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.495 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.495 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:31:09.495 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:31:09.495 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.495 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.495 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.495 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.495 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:31:09.496 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:31:09.496 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:31:09.496 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:31:09.496 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:31:09.496 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:31:09.496 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:31:09.497 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.497 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.497 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.497 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.497 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:31:09.497 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:31:09.497 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:31:09.498 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:31:09.498 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:31:09.498 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:31:09.498 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:31:09.498 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:09.498 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:31:09.498 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:31:09.498 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:31:09.498 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:31:09.498 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:31:09.498 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Teardown Test TEST_interval_changes ------------------ 2026-02-17T12:31:09.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_interval_changes ------------------' 2026-02-17T12:31:09.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T12:31:09.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:31:09.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:31:09.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:31:09.500 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:31:09.500 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:31:09.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:31:09.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:31:09.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:31:09.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:31:09.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:31:09.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:31:09.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:31:09.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:31:09.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:31:09.600 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:31:09.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:31:09.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:31:09.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:31:09.602 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:31:09.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:31:09.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:31:09.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:31:09.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:31:09.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:31:09.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:31:09.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:31:09.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:31:09.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:31:09.615 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Complete Test TEST_interval_changes ------------------ 2026-02-17T12:31:09.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_interval_changes ------------------' 2026-02-17T12:31:09.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T12:31:09.616 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Prepare Test TEST_just_deep_scrubs ------------------- 2026-02-17T12:31:09.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_just_deep_scrubs -------------------' 2026-02-17T12:31:09.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T12:31:09.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T12:31:09.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T12:31:09.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:31:09.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:31:09.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:31:09.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:31:09.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:31:09.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:31:09.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:31:09.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:31:09.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:31:09.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:31:09.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:31:09.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:31:09.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:31:09.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:31:09.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:31:09.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:31:09.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:31:09.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:31:09.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:31:09.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:31:09.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:31:09.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:31:09.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:31:09.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:31:09.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:31:09.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:31:09.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:31:09.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:31:09.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T12:31:09.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:31:09.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:31:09.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:31:09.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.508089 2026-02-17T12:31:09.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:31:09.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:31:09.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:31:09.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T12:31:09.631 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Run Test TEST_just_deep_scrubs ----------------------- 2026-02-17T12:31:09.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_just_deep_scrubs -----------------------' 2026-02-17T12:31:09.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_just_deep_scrubs td/osd-scrub-test 2026-02-17T12:31:09.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:467: TEST_just_deep_scrubs: local dir=td/osd-scrub-test 2026-02-17T12:31:09.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:468: TEST_just_deep_scrubs: cluster_conf=(['osds_num']='3' ['pgs_in_pool']='4' ['pool_name']='test') 2026-02-17T12:31:09.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:468: TEST_just_deep_scrubs: local -A cluster_conf 2026-02-17T12:31:09.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:474: TEST_just_deep_scrubs: standard_scrub_cluster td/osd-scrub-test cluster_conf 2026-02-17T12:31:09.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:229: standard_scrub_cluster: local dir=td/osd-scrub-test 2026-02-17T12:31:09.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:230: standard_scrub_cluster: local -n args=cluster_conf 2026-02-17T12:31:09.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:232: standard_scrub_cluster: local OSDS=3 2026-02-17T12:31:09.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:233: standard_scrub_cluster: local pg_num=4 2026-02-17T12:31:09.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:234: standard_scrub_cluster: local poolname=test 2026-02-17T12:31:09.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:235: standard_scrub_cluster: args['pool_name']=test 2026-02-17T12:31:09.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:236: standard_scrub_cluster: local extra_pars= 2026-02-17T12:31:09.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:237: standard_scrub_cluster: local debug_msg=dbg 2026-02-17T12:31:09.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:240: standard_scrub_cluster: local saved_echo_flag=x 2026-02-17T12:31:09.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:241: standard_scrub_cluster: set +x 2026-02-17T12:31:09.919 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 cfe47960-2121-4b56-83bb-5bee2ab41da7 2026-02-17T12:31:10.017 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:31:10.043 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:10.037+0000 7f8bcf3518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:10.045 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:10.037+0000 7f8bcf3518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:10.046 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:10.037+0000 7f8bcf3518c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:10.046 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:10.041+0000 7f8bcf3518c0 -1 bdev(0x555c37089c00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:31:10.046 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:10.041+0000 7f8bcf3518c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T12:31:12.836 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:31:13.035 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:31:13.059 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:13.053+0000 7f461a5678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:13.061 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:13.053+0000 7f461a5678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:13.062 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:13.057+0000 7f461a5678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:13.188 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:31:14.343 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:31:14.564 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:14.557+0000 7f461a5678c0 -1 Falling back to public interface 2026-02-17T12:31:15.501 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:31:16.656 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:31:17.611 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:17.605+0000 7f461a5678c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:31:17.811 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:31:18.967 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:31:19.118 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3423116271,v1:127.0.0.1:6803/3423116271] [v2:127.0.0.1:6804/3423116271,v1:127.0.0.1:6805/3423116271] exists,up cfe47960-2121-4b56-83bb-5bee2ab41da7 2026-02-17T12:31:19.123 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 7bb4e8ae-a2c0-4230-b24f-0d3776918e35 2026-02-17T12:31:19.289 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:31:19.316 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:19.309+0000 7f08075618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:19.317 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:19.309+0000 7f08075618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:19.318 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:19.313+0000 7f08075618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:19.319 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:19.313+0000 7f08075618c0 -1 bdev(0x55843a1cbc00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:31:19.319 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:19.313+0000 7f08075618c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T12:31:21.600 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:31:21.789 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:31:21.814 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:21.809+0000 7f4a9ad0a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:21.816 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:21.809+0000 7f4a9ad0a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:21.817 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:21.809+0000 7f4a9ad0a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:21.942 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:31:23.093 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:31:23.300 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:23.293+0000 7f4a9ad0a8c0 -1 Falling back to public interface 2026-02-17T12:31:24.244 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:31:25.392 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:31:26.310 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:26.301+0000 7f4a9ad0a8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:31:26.538 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:31:27.694 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:31:27.841 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1705751744,v1:127.0.0.1:6811/1705751744] [v2:127.0.0.1:6812/1705751744,v1:127.0.0.1:6813/1705751744] exists,up 7bb4e8ae-a2c0-4230-b24f-0d3776918e35 2026-02-17T12:31:27.846 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 67aace07-738c-4dcc-8262-9575a4085659 2026-02-17T12:31:28.015 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:31:28.043 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:28.037+0000 7ff637f6e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:28.045 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:28.037+0000 7ff637f6e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:28.046 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:28.037+0000 7ff637f6e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:28.047 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:28.041+0000 7ff637f6e8c0 -1 bdev(0x560b46b4bc00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:31:28.047 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:28.041+0000 7ff637f6e8c0 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-17T12:31:30.317 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T12:31:30.515 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T12:31:30.539 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:30.533+0000 7f9b6e4638c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:30.540 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:30.533+0000 7f9b6e4638c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:30.542 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:30.533+0000 7f9b6e4638c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:31:30.665 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:31:31.820 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:31:32.052 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:32.045+0000 7f9b6e4638c0 -1 Falling back to public interface 2026-02-17T12:31:32.974 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:31:34.128 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:31:35.089 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:31:35.081+0000 7f9b6e4638c0 -1 osd.2 0 log_to_monitors true 2026-02-17T12:31:35.284 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:31:36.442 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:31:36.593 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/619247351,v1:127.0.0.1:6819/619247351] [v2:127.0.0.1:6820/619247351,v1:127.0.0.1:6821/619247351] exists,up 67aace07-738c-4dcc-8262-9575a4085659 2026-02-17T12:31:36.794 INFO:tasks.workunit.client.0.trial128.stderr:pool 'test' created 2026-02-17T12:31:38.290 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836485 2026-02-17T12:31:38.446 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672964 2026-02-17T12:31:40.915 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 64424509442 2026-02-17T12:31:42.738 INFO:tasks.workunit.client.0.trial128.stdout:standard_scrub_cluster: dbg: test pool is test 1 2026-02-17T12:31:42.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:475: TEST_just_deep_scrubs: local poolid=1 2026-02-17T12:31:42.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:476: TEST_just_deep_scrubs: local poolname=test 2026-02-17T12:31:42.739 INFO:tasks.workunit.client.0.trial128.stdout:Pool: test : 1 2026-02-17T12:31:42.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:477: TEST_just_deep_scrubs: echo 'Pool: test : 1' 2026-02-17T12:31:42.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:479: TEST_just_deep_scrubs: TESTDATA=testdata.508089 2026-02-17T12:31:42.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:480: TEST_just_deep_scrubs: local objects=90 2026-02-17T12:31:42.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:481: TEST_just_deep_scrubs: dd if=/dev/urandom of=testdata.508089 bs=1032 count=1 2026-02-17T12:31:42.741 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records in 2026-02-17T12:31:42.741 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records out 2026-02-17T12:31:42.741 INFO:tasks.workunit.client.0.trial128.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 5.8679e-05 s, 17.6 MB/s 2026-02-17T12:31:42.741 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: seq 1 90 2026-02-17T12:31:42.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:42.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj1 testdata.508089 2026-02-17T12:31:42.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:42.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj2 testdata.508089 2026-02-17T12:31:42.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:42.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj3 testdata.508089 2026-02-17T12:31:42.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:42.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj4 testdata.508089 2026-02-17T12:31:42.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:42.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj5 testdata.508089 2026-02-17T12:31:42.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:42.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj6 testdata.508089 2026-02-17T12:31:42.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:42.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj7 testdata.508089 2026-02-17T12:31:42.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:42.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj8 testdata.508089 2026-02-17T12:31:42.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:42.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj9 testdata.508089 2026-02-17T12:31:42.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:42.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj10 testdata.508089 2026-02-17T12:31:42.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:42.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj11 testdata.508089 2026-02-17T12:31:42.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:42.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj12 testdata.508089 2026-02-17T12:31:42.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj13 testdata.508089 2026-02-17T12:31:43.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj14 testdata.508089 2026-02-17T12:31:43.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj15 testdata.508089 2026-02-17T12:31:43.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj16 testdata.508089 2026-02-17T12:31:43.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj17 testdata.508089 2026-02-17T12:31:43.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj18 testdata.508089 2026-02-17T12:31:43.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj19 testdata.508089 2026-02-17T12:31:43.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj20 testdata.508089 2026-02-17T12:31:43.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj21 testdata.508089 2026-02-17T12:31:43.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj22 testdata.508089 2026-02-17T12:31:43.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.215 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj23 testdata.508089 2026-02-17T12:31:43.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj24 testdata.508089 2026-02-17T12:31:43.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj25 testdata.508089 2026-02-17T12:31:43.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj26 testdata.508089 2026-02-17T12:31:43.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj27 testdata.508089 2026-02-17T12:31:43.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj28 testdata.508089 2026-02-17T12:31:43.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj29 testdata.508089 2026-02-17T12:31:43.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj30 testdata.508089 2026-02-17T12:31:43.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj31 testdata.508089 2026-02-17T12:31:43.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj32 testdata.508089 2026-02-17T12:31:43.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj33 testdata.508089 2026-02-17T12:31:43.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj34 testdata.508089 2026-02-17T12:31:43.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj35 testdata.508089 2026-02-17T12:31:43.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj36 testdata.508089 2026-02-17T12:31:43.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj37 testdata.508089 2026-02-17T12:31:43.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj38 testdata.508089 2026-02-17T12:31:43.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj39 testdata.508089 2026-02-17T12:31:43.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj40 testdata.508089 2026-02-17T12:31:43.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj41 testdata.508089 2026-02-17T12:31:43.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj42 testdata.508089 2026-02-17T12:31:43.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj43 testdata.508089 2026-02-17T12:31:43.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj44 testdata.508089 2026-02-17T12:31:43.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj45 testdata.508089 2026-02-17T12:31:43.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj46 testdata.508089 2026-02-17T12:31:43.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj47 testdata.508089 2026-02-17T12:31:43.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj48 testdata.508089 2026-02-17T12:31:43.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj49 testdata.508089 2026-02-17T12:31:43.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj50 testdata.508089 2026-02-17T12:31:43.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj51 testdata.508089 2026-02-17T12:31:43.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj52 testdata.508089 2026-02-17T12:31:43.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj53 testdata.508089 2026-02-17T12:31:43.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj54 testdata.508089 2026-02-17T12:31:43.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj55 testdata.508089 2026-02-17T12:31:43.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj56 testdata.508089 2026-02-17T12:31:43.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:43.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj57 testdata.508089 2026-02-17T12:31:44.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj58 testdata.508089 2026-02-17T12:31:44.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj59 testdata.508089 2026-02-17T12:31:44.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj60 testdata.508089 2026-02-17T12:31:44.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj61 testdata.508089 2026-02-17T12:31:44.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj62 testdata.508089 2026-02-17T12:31:44.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj63 testdata.508089 2026-02-17T12:31:44.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj64 testdata.508089 2026-02-17T12:31:44.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj65 testdata.508089 2026-02-17T12:31:44.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj66 testdata.508089 2026-02-17T12:31:44.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj67 testdata.508089 2026-02-17T12:31:44.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj68 testdata.508089 2026-02-17T12:31:44.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj69 testdata.508089 2026-02-17T12:31:44.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj70 testdata.508089 2026-02-17T12:31:44.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj71 testdata.508089 2026-02-17T12:31:44.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj72 testdata.508089 2026-02-17T12:31:44.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj73 testdata.508089 2026-02-17T12:31:44.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj74 testdata.508089 2026-02-17T12:31:44.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj75 testdata.508089 2026-02-17T12:31:44.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj76 testdata.508089 2026-02-17T12:31:44.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj77 testdata.508089 2026-02-17T12:31:44.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj78 testdata.508089 2026-02-17T12:31:44.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj79 testdata.508089 2026-02-17T12:31:44.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj80 testdata.508089 2026-02-17T12:31:44.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj81 testdata.508089 2026-02-17T12:31:44.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj82 testdata.508089 2026-02-17T12:31:44.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj83 testdata.508089 2026-02-17T12:31:44.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj84 testdata.508089 2026-02-17T12:31:44.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj85 testdata.508089 2026-02-17T12:31:44.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj86 testdata.508089 2026-02-17T12:31:44.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj87 testdata.508089 2026-02-17T12:31:44.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj88 testdata.508089 2026-02-17T12:31:44.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj89 testdata.508089 2026-02-17T12:31:44.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:482: TEST_just_deep_scrubs: for i in `seq 1 $objects` 2026-02-17T12:31:44.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:484: TEST_just_deep_scrubs: rados -p test put obj90 testdata.508089 2026-02-17T12:31:44.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:486: TEST_just_deep_scrubs: rm -f testdata.508089 2026-02-17T12:31:44.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:491: TEST_just_deep_scrubs: ceph osd set noscrub 2026-02-17T12:31:44.952 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:31:44.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:492: TEST_just_deep_scrubs: ceph osd set nodeep-scrub 2026-02-17T12:31:45.157 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is set 2026-02-17T12:31:45.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:493: TEST_just_deep_scrubs: sleep 6 2026-02-17T12:31:51.169 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:494: TEST_just_deep_scrubs: date -Ins 2026-02-17T12:31:51.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:494: TEST_just_deep_scrubs: local now_is=2026-02-17T12:31:51,166479039+00:00 2026-02-17T12:31:51.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:495: TEST_just_deep_scrubs: declare -A sched_data 2026-02-17T12:31:51.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:496: TEST_just_deep_scrubs: local pgid=1.2 2026-02-17T12:31:51.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:499: TEST_just_deep_scrubs: set_query_debug 1.2 2026-02-17T12:31:51.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:303: set_query_debug: local pgid=1.2 2026-02-17T12:31:51.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:305: set_query_debug: ceph pg dump pgs_brief 2026-02-17T12:31:51.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:305: set_query_debug: awk -v 'pg=^1.2' -n -e '$0 ~ pg { print(gensub(/[^0-9]*([0-9]+).*/,"\\1","g",$5)); }' 2026-02-17T12:31:51.313 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs_brief 2026-02-17T12:31:51.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:305: set_query_debug: local prim_osd=0 2026-02-17T12:31:51.323 INFO:tasks.workunit.client.0.trial128.stdout:Setting scrub debug data. Primary for 1.2 is 0 2026-02-17T12:31:51.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:307: set_query_debug: echo 'Setting scrub debug data. Primary for 1.2 is 0' 2026-02-17T12:31:51.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:308: set_query_debug: get_asok_path osd.0 2026-02-17T12:31:51.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-17T12:31:51.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-17T12:31:51.325 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:31:51.325 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:31:51.325 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:31:51.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-osd.0.asok 2026-02-17T12:31:51.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:308: set_query_debug: CEPH_ARGS= 2026-02-17T12:31:51.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:308: set_query_debug: ceph --format=json daemon /tmp/ceph-asok.508089/ceph-osd.0.asok scrubdebug 1.2 set sessions 2026-02-17T12:31:51.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:501: TEST_just_deep_scrubs: extract_published_sch 1.2 2026-02-17T12:31:51,166479039+00:00 2026-02-17T12:31:51,166479039+00:00 sched_data 2026-02-17T12:31:51.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:17: extract_published_sch: local pgn=1.2 2026-02-17T12:31:51.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:18: extract_published_sch: local -n dict=sched_data 2026-02-17T12:31:51.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:19: extract_published_sch: local current_time=2026-02-17T12:31:51,166479039+00:00 2026-02-17T12:31:51.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:20: extract_published_sch: local extra_time=2026-02-17T12:31:51,166479039+00:00 2026-02-17T12:31:51.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:21: extract_published_sch: local extr_dbg=2 2026-02-17T12:31:51.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:24: extract_published_sch: local saved_echo_flag=x 2026-02-17T12:31:51.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:25: extract_published_sch: set +x 2026-02-17T12:31:51.537 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:31:51.548 INFO:tasks.workunit.client.0.trial128.stdout:{"success":true}from pg dump pg: { 2026-02-17T12:31:51.548 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_pg_state": "active+clean", 2026-02-17T12:31:51.548 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T12:31:51.548 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_last_duration": 0, 2026-02-17T12:31:51.549 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule": "periodic scrub scheduled", 2026-02-17T12:31:51.549 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule_at": "2026-02-18T12:31:36.789234+0000", 2026-02-17T12:31:51.549 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_is_future": true, 2026-02-17T12:31:51.549 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_vs_date": true, 2026-02-17T12:31:51.549 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_reported_epoch": 20, 2026-02-17T12:31:51.549 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_seq": 36 2026-02-17T12:31:51.549 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:31:51.549 INFO:tasks.workunit.client.0.trial128.stdout:query output: 2026-02-17T12:31:51.620 INFO:tasks.workunit.client.0.trial128.stdout: "scrubber": { 2026-02-17T12:31:51.620 INFO:tasks.workunit.client.0.trial128.stdout: "active": false, 2026-02-17T12:31:51.620 INFO:tasks.workunit.client.0.trial128.stdout: "must_scrub": false, 2026-02-17T12:31:51.621 INFO:tasks.workunit.client.0.trial128.stdout: "must_deep_scrub": false, 2026-02-17T12:31:51.621 INFO:tasks.workunit.client.0.trial128.stdout: "must_repair": false, 2026-02-17T12:31:51.621 INFO:tasks.workunit.client.0.trial128.stdout: "need_auto": false, 2026-02-17T12:31:51.621 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reg_stamp": "2026-02-18T12:31:36.789234+0000", 2026-02-17T12:31:51.621 INFO:tasks.workunit.client.0.trial128.stdout: "schedule": "scrub scheduled @ 2026-02-18T12:31:36.789 (2026-02-18T12:31:36.789)", 2026-02-17T12:31:51.621 INFO:tasks.workunit.client.0.trial128.stdout: "test_sequence": 5976576 2026-02-17T12:31:51.621 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:51.621 INFO:tasks.workunit.client.0.trial128.stdout: "agent_state": {} 2026-02-17T12:31:51.702 INFO:tasks.workunit.client.0.trial128.stdout:( 2026-02-17T12:31:51.702 INFO:tasks.workunit.client.0.trial128.stdout:[query_epoch]=20 2026-02-17T12:31:51.702 INFO:tasks.workunit.client.0.trial128.stdout:[query_seq]=37 2026-02-17T12:31:51.702 INFO:tasks.workunit.client.0.trial128.stdout:[query_active]=false 2026-02-17T12:31:51.702 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule]='scrub scheduled' 2026-02-17T12:31:51.702 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule_at]='2026-02-18T12:31:36.789' 2026-02-17T12:31:51.702 INFO:tasks.workunit.client.0.trial128.stdout:[query_target_at]='2026-02-18T12:31:36.789' 2026-02-17T12:31:51.702 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_duration]=0 2026-02-17T12:31:51.702 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_stamp]='2026-02-17T12:31:36.789234+0000' 2026-02-17T12:31:51.702 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_scrub]='0x0' 2026-02-17T12:31:51.702 INFO:tasks.workunit.client.0.trial128.stdout:[query_is_future]=true 2026-02-17T12:31:51.702 INFO:tasks.workunit.client.0.trial128.stdout:[query_vs_date]=true 2026-02-17T12:31:51.702 INFO:tasks.workunit.client.0.trial128.stdout:[query_scrub_seq]=5976576 2026-02-17T12:31:51.703 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_pg_state]='active+clean' 2026-02-17T12:31:51.703 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T12:31:51.703 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_last_duration]=0 2026-02-17T12:31:51.703 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule]='periodic scrub scheduled' 2026-02-17T12:31:51.703 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule_at]='2026-02-18T12:31:36.789234+0000' 2026-02-17T12:31:51.703 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_is_future]=true 2026-02-17T12:31:51.703 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_vs_date]=true 2026-02-17T12:31:51.703 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_reported_epoch]=20 2026-02-17T12:31:51.703 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_seq]=36 2026-02-17T12:31:51.703 INFO:tasks.workunit.client.0.trial128.stdout:) 2026-02-17T12:31:51.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:502: TEST_just_deep_scrubs: local saved_last_stamp=2026-02-17T12:31:36.789234+0000 2026-02-17T12:31:51.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:503: TEST_just_deep_scrubs: local dbg_counter_at_start=5976576 2026-02-17T12:31:51.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:504: TEST_just_deep_scrubs: echo 'test counter @ start: 5976576' 2026-02-17T12:31:51.715 INFO:tasks.workunit.client.0.trial128.stdout:test counter @ start: 5976576 2026-02-17T12:31:51.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:506: TEST_just_deep_scrubs: ceph tell 1.2 schedule-deep-scrub 2026-02-17T12:31:51.776 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:31:51.776 INFO:tasks.workunit.client.0.trial128.stdout: "deep": true, 2026-02-17T12:31:51.776 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:31:51.776 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-03T12:30:11.771417+0000" 2026-02-17T12:31:51.776 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:31:51.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:508: TEST_just_deep_scrubs: sleep 5 2026-02-17T12:31:56.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:510: TEST_just_deep_scrubs: ceph pg dump pgs --format=json-pretty 2026-02-17T12:31:56.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:510: TEST_just_deep_scrubs: jq -r '.pg_stats[] | "\(.pgid) \(.stat_sum.num_objects)"' 2026-02-17T12:31:56.931 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:31:56.940 INFO:tasks.workunit.client.0.trial128.stdout:1.3 20 2026-02-17T12:31:56.940 INFO:tasks.workunit.client.0.trial128.stdout:1.2 20 2026-02-17T12:31:56.940 INFO:tasks.workunit.client.0.trial128.stdout:1.1 25 2026-02-17T12:31:56.940 INFO:tasks.workunit.client.0.trial128.stdout:1.0 25 2026-02-17T12:31:56.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:511: TEST_just_deep_scrubs: ceph pg 1.2 query --format=json-pretty 2026-02-17T12:31:56.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:511: TEST_just_deep_scrubs: jq -r .info.stats.stat_sum.num_objects 2026-02-17T12:31:57.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:511: TEST_just_deep_scrubs: echo 'Objects # in pg 1.2: ' 20 2026-02-17T12:31:57.014 INFO:tasks.workunit.client.0.trial128.stdout:Objects # in pg 1.2: 20 2026-02-17T12:31:57.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:513: TEST_just_deep_scrubs: declare -A sc_data_2 2026-02-17T12:31:57.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:514: TEST_just_deep_scrubs: extract_published_sch 1.2 2026-02-17T12:31:51,166479039+00:00 2026-02-17T12:31:51,166479039+00:00 sc_data_2 2026-02-17T12:31:57.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:17: extract_published_sch: local pgn=1.2 2026-02-17T12:31:57.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:18: extract_published_sch: local -n dict=sc_data_2 2026-02-17T12:31:57.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:19: extract_published_sch: local current_time=2026-02-17T12:31:51,166479039+00:00 2026-02-17T12:31:57.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:20: extract_published_sch: local extra_time=2026-02-17T12:31:51,166479039+00:00 2026-02-17T12:31:57.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:21: extract_published_sch: local extr_dbg=2 2026-02-17T12:31:57.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:24: extract_published_sch: local saved_echo_flag=x 2026-02-17T12:31:57.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:25: extract_published_sch: set +x 2026-02-17T12:31:57.159 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:31:57.168 INFO:tasks.workunit.client.0.trial128.stdout:from pg dump pg: { 2026-02-17T12:31:57.169 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_pg_state": "active+clean", 2026-02-17T12:31:57.169 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T12:31:57.169 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_last_duration": 0, 2026-02-17T12:31:57.169 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule": "queued for deep scrub", 2026-02-17T12:31:57.169 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule_at": "0", 2026-02-17T12:31:57.169 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_is_future": false, 2026-02-17T12:31:57.169 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_vs_date": false, 2026-02-17T12:31:57.169 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_reported_epoch": 20, 2026-02-17T12:31:57.169 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_seq": 38 2026-02-17T12:31:57.169 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:31:57.169 INFO:tasks.workunit.client.0.trial128.stdout:query output: 2026-02-17T12:31:57.239 INFO:tasks.workunit.client.0.trial128.stdout: "scrubber": { 2026-02-17T12:31:57.239 INFO:tasks.workunit.client.0.trial128.stdout: "active": false, 2026-02-17T12:31:57.239 INFO:tasks.workunit.client.0.trial128.stdout: "must_scrub": false, 2026-02-17T12:31:57.239 INFO:tasks.workunit.client.0.trial128.stdout: "must_deep_scrub": false, 2026-02-17T12:31:57.239 INFO:tasks.workunit.client.0.trial128.stdout: "must_repair": false, 2026-02-17T12:31:57.239 INFO:tasks.workunit.client.0.trial128.stdout: "need_auto": false, 2026-02-17T12:31:57.239 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reg_stamp": "2026-02-17T12:32:02.150079+0000", 2026-02-17T12:31:57.239 INFO:tasks.workunit.client.0.trial128.stdout: "schedule": "deep scrub scheduled @ 2026-02-17T12:32:02.150 (2026-02-04T12:30:11.771)", 2026-02-17T12:31:57.239 INFO:tasks.workunit.client.0.trial128.stdout: "test_sequence": 5976576 2026-02-17T12:31:57.239 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:31:57.240 INFO:tasks.workunit.client.0.trial128.stdout: "agent_state": {} 2026-02-17T12:31:57.323 INFO:tasks.workunit.client.0.trial128.stdout:( 2026-02-17T12:31:57.323 INFO:tasks.workunit.client.0.trial128.stdout:[query_epoch]=20 2026-02-17T12:31:57.323 INFO:tasks.workunit.client.0.trial128.stdout:[query_seq]=39 2026-02-17T12:31:57.323 INFO:tasks.workunit.client.0.trial128.stdout:[query_active]=false 2026-02-17T12:31:57.323 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule]='deep scrub scheduled' 2026-02-17T12:31:57.324 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule_at]='2026-02-17T12:32:02.150' 2026-02-17T12:31:57.324 INFO:tasks.workunit.client.0.trial128.stdout:[query_target_at]='2026-02-04T12:30:11.771' 2026-02-17T12:31:57.324 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_duration]=0 2026-02-17T12:31:57.324 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_stamp]='2026-02-03T12:30:11.771417+0000' 2026-02-17T12:31:57.324 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_scrub]='0x0' 2026-02-17T12:31:57.324 INFO:tasks.workunit.client.0.trial128.stdout:[query_is_future]=false 2026-02-17T12:31:57.324 INFO:tasks.workunit.client.0.trial128.stdout:[query_vs_date]=true 2026-02-17T12:31:57.324 INFO:tasks.workunit.client.0.trial128.stdout:[query_scrub_seq]=5976576 2026-02-17T12:31:57.324 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_pg_state]='active+clean' 2026-02-17T12:31:57.324 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T12:31:57.324 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_last_duration]=0 2026-02-17T12:31:57.324 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule]='queued for deep scrub' 2026-02-17T12:31:57.325 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule_at]='0' 2026-02-17T12:31:57.325 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_is_future]=false 2026-02-17T12:31:57.325 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_vs_date]=false 2026-02-17T12:31:57.325 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_reported_epoch]=20 2026-02-17T12:31:57.325 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_seq]=38 2026-02-17T12:31:57.325 INFO:tasks.workunit.client.0.trial128.stdout:) 2026-02-17T12:31:57.334 INFO:tasks.workunit.client.0.trial128.stdout:test counter @ should show no change: 5976576 2026-02-17T12:31:57.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:515: TEST_just_deep_scrubs: echo 'test counter @ should show no change: ' 5976576 2026-02-17T12:31:57.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:516: TEST_just_deep_scrubs: (( 0 == 0 )) 2026-02-17T12:31:57.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:517: TEST_just_deep_scrubs: (( 5976576 == 5976576 )) 2026-02-17T12:31:57.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:520: TEST_just_deep_scrubs: ceph osd unset nodeep-scrub 2026-02-17T12:31:57.534 INFO:tasks.workunit.client.0.trial128.stderr:nodeep-scrub is unset 2026-02-17T12:31:57.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:521: TEST_just_deep_scrubs: sleep 5 2026-02-17T12:32:02.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:522: TEST_just_deep_scrubs: expct_qry_duration=(['query_last_duration']='0' ['query_last_duration_neg']='not0') 2026-02-17T12:32:02.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:522: TEST_just_deep_scrubs: declare -A expct_qry_duration 2026-02-17T12:32:02.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:523: TEST_just_deep_scrubs: sc_data_2=() 2026-02-17T12:32:02.547 INFO:tasks.workunit.client.0.trial128.stdout:test counter @ should be higher than before the unset: 2026-02-17T12:32:02.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:524: TEST_just_deep_scrubs: echo 'test counter @ should be higher than before the unset: ' 2026-02-17T12:32:02.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:525: TEST_just_deep_scrubs: wait_any_cond 1.2 10 2026-02-17T12:31:36.789234+0000 expct_qry_duration 'WaitingAfterScrub ' sc_data_2 2026-02-17T12:32:02.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:106: wait_any_cond: local pgid=1.2 2026-02-17T12:32:02.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:107: wait_any_cond: local retries=10 2026-02-17T12:32:02.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:108: wait_any_cond: local cmp_date=2026-02-17T12:31:36.789234+0000 2026-02-17T12:32:02.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:109: wait_any_cond: local -n ep=expct_qry_duration 2026-02-17T12:32:02.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:110: wait_any_cond: local -n out_array=sc_data_2 2026-02-17T12:32:02.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:111: wait_any_cond: local -A sc_data 2026-02-17T12:32:02.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:112: wait_any_cond: local extr_dbg=2 2026-02-17T12:32:02.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:115: wait_any_cond: local saved_echo_flag=x 2026-02-17T12:32:02.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:116: wait_any_cond: set +x 2026-02-17T12:32:02.549 INFO:tasks.workunit.client.0.trial128.stdout:waiting for any condition (WaitingAfterScrub ): pg:1.2 dt:2026-02-17T12:31:36.789234+0000 (10 retries) 2026-02-17T12:32:03.193 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:32:03.203 INFO:tasks.workunit.client.0.trial128.stdout:from pg dump pg: { 2026-02-17T12:32:03.203 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_pg_state": "active+clean", 2026-02-17T12:32:03.203 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_state_has_scrubbing": false, 2026-02-17T12:32:03.203 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_last_duration": 0, 2026-02-17T12:32:03.203 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule": "periodic deep scrub scheduled", 2026-02-17T12:32:03.203 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_schedule_at": "2026-02-17T12:32:02.150079+0000", 2026-02-17T12:32:03.203 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_is_future": true, 2026-02-17T12:32:03.203 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_vs_date": true, 2026-02-17T12:32:03.203 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_reported_epoch": 21, 2026-02-17T12:32:03.204 INFO:tasks.workunit.client.0.trial128.stdout: "dmp_seq": 40 2026-02-17T12:32:03.204 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:32:03.204 INFO:tasks.workunit.client.0.trial128.stdout:query output: 2026-02-17T12:32:03.276 INFO:tasks.workunit.client.0.trial128.stdout: "scrubber": { 2026-02-17T12:32:03.276 INFO:tasks.workunit.client.0.trial128.stdout: "active": false, 2026-02-17T12:32:03.276 INFO:tasks.workunit.client.0.trial128.stdout: "must_scrub": false, 2026-02-17T12:32:03.276 INFO:tasks.workunit.client.0.trial128.stdout: "must_deep_scrub": false, 2026-02-17T12:32:03.276 INFO:tasks.workunit.client.0.trial128.stdout: "must_repair": false, 2026-02-17T12:32:03.276 INFO:tasks.workunit.client.0.trial128.stdout: "need_auto": false, 2026-02-17T12:32:03.276 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reg_stamp": "2026-02-18T12:32:03.176735+0000", 2026-02-17T12:32:03.276 INFO:tasks.workunit.client.0.trial128.stdout: "schedule": "scrub scheduled @ 2026-02-18T12:32:03.176 (2026-02-18T12:32:03.176)", 2026-02-17T12:32:03.276 INFO:tasks.workunit.client.0.trial128.stdout: "test_sequence": 5976579 2026-02-17T12:32:03.276 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.276 INFO:tasks.workunit.client.0.trial128.stdout: "agent_state": {} 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:( 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[query_epoch]=21 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[query_seq]=50 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[query_active]=false 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule]='scrub scheduled' 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[query_schedule_at]='2026-02-18T12:32:03.176' 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[query_target_at]='2026-02-18T12:32:03.176' 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_duration]=1 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_stamp]='2026-02-17T12:32:03.176735+0000' 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[query_last_scrub]='18x20' 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[query_is_future]=true 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[query_vs_date]=true 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[query_scrub_seq]=5976579 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_pg_state]='active+clean' 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_state_has_scrubbing]=false 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_last_duration]=0 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule]='periodic deep scrub scheduled' 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_schedule_at]='2026-02-17T12:32:02.150079+0000' 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_is_future]=true 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_vs_date]=true 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_reported_epoch]=21 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:[dmp_seq]=40 2026-02-17T12:32:03.358 INFO:tasks.workunit.client.0.trial128.stdout:) 2026-02-17T12:32:03.370 INFO:tasks.workunit.client.0.trial128.stdout:--> loop: 1 ~ false / 50 / 40 / true / 2026-02-17T12:32:03.176735+0000 / scrub scheduled %%% query_last_duration_neg query_last_duration 2026-02-17T12:32:03.370 INFO:tasks.workunit.client.0.trial128.stdout:key is query_last_duration: negation:1 # expected: 0 # in actual: 1 2026-02-17T12:32:03.370 INFO:tasks.workunit.client.0.trial128.stdout:WaitingAfterScrub - 'query_last_duration' actual value (1) matches expected (0) (negation: 1) 2026-02-17T12:32:03.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/scrub-helpers.sh:157: wait_any_cond: return 0 2026-02-17T12:32:03.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:526: TEST_just_deep_scrubs: perf_counters td/osd-scrub-test 3 2026-02-17T12:32:03.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-17T12:32:03.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-17T12:32:03.371 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-17T12:32:03.372 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-17T12:32:03.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:32:03.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-17T12:32:03.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:32:03.444 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:32:03.445 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:32:03.445 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:03.445 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:03.445 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:32:03.445 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:32:03.446 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.446 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:03.446 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:03.446 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:03.446 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:03.446 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:03.447 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.447 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.447 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.447 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.447 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:03.448 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:03.448 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.448 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.448 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.448 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.448 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:03.449 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:03.449 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:03.449 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:03.449 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:03.449 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:03.450 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:03.450 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.450 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.450 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.450 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.450 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:03.451 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:03.451 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:03.451 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:03.451 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.451 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.451 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.452 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.452 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:03.452 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.452 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.452 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:03.453 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:32:03.453 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:03.453 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:03.453 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:32:03.453 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:32:03.453 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.454 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:03.454 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 1, 2026-02-17T12:32:03.454 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 1, 2026-02-17T12:32:03.454 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 1, 2026-02-17T12:32:03.454 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:03.454 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 1, 2026-02-17T12:32:03.455 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0.003999926, 2026-02-17T12:32:03.455 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0.003999926 2026-02-17T12:32:03.455 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.455 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:03.455 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:03.456 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.456 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.456 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.456 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.456 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:03.456 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 2, 2026-02-17T12:32:03.457 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:03.457 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:03.457 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:03.457 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 1, 2026-02-17T12:32:03.457 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:03.458 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 1, 2026-02-17T12:32:03.458 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.458 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.458 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.458 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:03.458 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:03.459 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:03.459 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:03.459 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.459 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.459 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.459 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.460 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 2 2026-02-17T12:32:03.460 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.460 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.460 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:03.460 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:32:03.461 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:03.461 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:03.461 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:32:03.461 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:32:03.461 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.461 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:03.462 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:03.462 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:03.462 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:03.462 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:03.462 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.462 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.463 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.463 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.463 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:03.463 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:03.463 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.463 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.464 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.464 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.464 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:03.464 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:03.464 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:03.464 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:03.465 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:03.465 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:03.465 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:03.465 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.465 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.465 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.465 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.466 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:03.466 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:03.466 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:03.466 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:03.466 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.466 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.466 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.467 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.467 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:03.467 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.467 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.467 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:03.467 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:32:03.467 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:03.468 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:03.468 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:32:03.468 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:32:03.468 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.468 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:03.468 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:03.468 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:03.469 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:03.469 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:03.469 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.469 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.469 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.469 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.469 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:03.469 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:03.470 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.470 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.470 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.470 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.470 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:03.470 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:03.470 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:03.470 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:03.471 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:03.471 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:03.471 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:03.472 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.472 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.472 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.472 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.472 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:03.472 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:03.472 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:03.472 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:03.473 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.473 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.473 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.473 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.473 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:03.473 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.473 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.473 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:32:03.474 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:32:03.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:32:03.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-17T12:32:03.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:32:03.512 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:32:03.512 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:32:03.512 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:03.512 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:03.512 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:32:03.512 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:32:03.512 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.512 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:03.512 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:03.513 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:03.514 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:03.515 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:32:03.516 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:03.517 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.518 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.519 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.520 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.520 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:03.520 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:03.520 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:03.520 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:03.520 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:03.520 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:03.520 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:03.520 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.520 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.520 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.520 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.520 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:03.520 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:03.521 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:03.521 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:03.521 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.521 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.521 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.521 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.521 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:03.521 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.521 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.521 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:32:03.521 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:32:03.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:32:03.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-17T12:32:03.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:32:03.580 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:32:03.580 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:32:03.581 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:03.581 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:03.581 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:32:03.581 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:32:03.581 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.581 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:03.581 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:03.581 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:03.581 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:03.582 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:03.582 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.582 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.582 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.582 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.582 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:03.582 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:03.582 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.582 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.582 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.582 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.583 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:03.583 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:03.583 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:03.583 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:03.583 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:03.583 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:03.583 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:03.583 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.583 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.583 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.583 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.583 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:03.584 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:03.584 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:03.584 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:03.584 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.584 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.584 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.584 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.584 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:03.584 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.584 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.584 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:03.584 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:32:03.585 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:03.585 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:03.585 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:32:03.585 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:32:03.585 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.585 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:03.585 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:03.585 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:03.585 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:03.585 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:03.585 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.585 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.585 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.586 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.586 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:03.586 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:03.586 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.586 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.586 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.586 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.586 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:03.586 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:03.586 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:03.586 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:03.586 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:03.587 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:03.587 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:03.587 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.587 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.587 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.587 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.587 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:03.587 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:03.587 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:03.587 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:03.587 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.587 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.588 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.588 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.588 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:03.588 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.588 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.588 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:03.588 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:32:03.588 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:03.588 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:03.588 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:32:03.588 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:32:03.588 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.588 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:03.589 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:03.589 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:03.589 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:03.589 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:03.589 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.589 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.589 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.589 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.589 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:03.589 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:03.589 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.590 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.590 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.590 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.590 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:03.590 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:03.590 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:03.590 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:03.590 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:03.590 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:03.590 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:03.591 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.591 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.591 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.591 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.591 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:03.591 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:03.591 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:03.591 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:03.591 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.591 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.592 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.592 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.592 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:03.592 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.592 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.592 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:03.592 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:32:03.592 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:03.592 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:03.592 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:32:03.593 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:32:03.593 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.593 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:03.593 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:03.593 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:03.593 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:03.593 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:03.593 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.593 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.593 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.594 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.594 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:03.594 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:03.594 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.594 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.594 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.594 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.594 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:03.594 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:03.594 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:03.594 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:03.595 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:03.595 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:03.595 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:03.596 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.596 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.596 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.596 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.596 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:03.596 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:03.596 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:03.596 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:03.596 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:03.596 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:03.597 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:03.597 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:03.597 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:03.597 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.597 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:03.597 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:32:03.597 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:32:03.597 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Teardown Test TEST_just_deep_scrubs ------------------ 2026-02-17T12:32:03.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_just_deep_scrubs ------------------' 2026-02-17T12:32:03.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T12:32:03.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:32:03.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:32:03.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:32:03.599 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:32:03.599 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:32:03.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:32:03.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:32:03.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:32:03.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:32:03.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:32:03.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:32:03.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:32:03.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:32:03.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:32:03.903 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:32:03.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:32:03.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:32:03.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:32:03.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:32:03.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:32:03.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:32:03.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:32:03.920 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:32:03.920 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:03.920 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:03.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:32:03.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:32:03.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:32:03.922 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Complete Test TEST_just_deep_scrubs ------------------ 2026-02-17T12:32:03.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_just_deep_scrubs ------------------' 2026-02-17T12:32:03.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T12:32:03.923 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Prepare Test TEST_pg_dump_objects_scrubbed ------------------- 2026-02-17T12:32:03.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_pg_dump_objects_scrubbed -------------------' 2026-02-17T12:32:03.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T12:32:03.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T12:32:03.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T12:32:03.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:32:03.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:32:03.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:32:03.924 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:32:03.924 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:32:03.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:32:03.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:32:03.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:32:03.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:32:03.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:32:03.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:32:03.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:32:03.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:32:03.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:32:03.929 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:32:03.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:32:03.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:32:03.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:32:03.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:32:03.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:32:03.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:32:03.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:32:03.933 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:32:03.934 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:03.934 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:03.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:32:03.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:32:03.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:32:03.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T12:32:03.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:32:03.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:03.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:03.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.508089 2026-02-17T12:32:03.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:32:03.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:32:03.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:32:03.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T12:32:03.938 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Run Test TEST_pg_dump_objects_scrubbed ----------------------- 2026-02-17T12:32:03.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_pg_dump_objects_scrubbed -----------------------' 2026-02-17T12:32:03.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_pg_dump_objects_scrubbed td/osd-scrub-test 2026-02-17T12:32:03.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:651: TEST_pg_dump_objects_scrubbed: local dir=td/osd-scrub-test 2026-02-17T12:32:03.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:652: TEST_pg_dump_objects_scrubbed: local poolname=test 2026-02-17T12:32:03.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:653: TEST_pg_dump_objects_scrubbed: local OSDS=3 2026-02-17T12:32:03.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:654: TEST_pg_dump_objects_scrubbed: local objects=15 2026-02-17T12:32:03.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:655: TEST_pg_dump_objects_scrubbed: local timeout=10 2026-02-17T12:32:03.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:657: TEST_pg_dump_objects_scrubbed: TESTDATA=testdata.508089 2026-02-17T12:32:03.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:659: TEST_pg_dump_objects_scrubbed: setup td/osd-scrub-test 2026-02-17T12:32:03.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T12:32:03.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T12:32:03.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:32:03.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:32:03.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:32:03.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:32:03.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:32:03.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:32:03.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:32:03.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:32:03.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:32:03.942 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:32:03.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:32:03.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:32:03.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:32:03.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:32:03.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:32:03.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:32:03.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:32:03.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:32:03.946 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:32:03.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:32:03.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:32:03.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:32:03.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:32:03.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:03.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:03.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:32:03.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:32:03.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:32:03.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T12:32:03.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:32:03.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:03.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:03.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.508089 2026-02-17T12:32:03.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:32:03.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:32:03.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:32:03.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T12:32:03.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:660: TEST_pg_dump_objects_scrubbed: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-17T12:32:03.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-test 2026-02-17T12:32:03.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:32:03.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:32:03.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:32:03.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-test/a 2026-02-17T12:32:03.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-17T12:32:03.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:32:03.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:32:03.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:32:03.978 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:32:03.978 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:03.978 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:03.978 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:32:03.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-17T12:32:04.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:32:04.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:32:04.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:32:04.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:32:04.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:32:04.007 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:32:04.007 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:32:04.007 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:32:04.007 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:32:04.008 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:32:04.008 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:04.008 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:04.008 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-mon.a.asok 2026-02-17T12:32:04.008 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:32:04.008 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.508089/ceph-mon.a.asok config get fsid 2026-02-17T12:32:04.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:32:04.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:32:04.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:32:04.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:32:04.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:32:04.065 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:32:04.065 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:32:04.065 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:32:04.065 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:32:04.065 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:04.065 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:04.065 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-mon.a.asok 2026-02-17T12:32:04.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:32:04.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.508089/ceph-mon.a.asok config get mon_host 2026-02-17T12:32:04.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:661: TEST_pg_dump_objects_scrubbed: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-17T12:32:04.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-test 2026-02-17T12:32:04.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:32:04.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:32:04.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:32:04.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-test/x 2026-02-17T12:32:04.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:32:04.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:32:04.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:32:04.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:32:04.211 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:32:04.211 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:04.211 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:04.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:32:04.211 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:32:04.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T12:32:04.230 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:662: TEST_pg_dump_objects_scrubbed: expr 3 - 1 2026-02-17T12:32:04.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:662: TEST_pg_dump_objects_scrubbed: seq 0 2 2026-02-17T12:32:04.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:662: TEST_pg_dump_objects_scrubbed: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:32:04.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:664: TEST_pg_dump_objects_scrubbed: run_osd td/osd-scrub-test 0 2026-02-17T12:32:04.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:32:04.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:32:04.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:32:04.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:32:04.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-17T12:32:04.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:32:04.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:32:04.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:32:04.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:32:04.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-17T12:32:04.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-17T12:32:04.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:32:04.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:32:04.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:32:04.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:32:04.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:32:04.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:32:04.237 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:32:04.237 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:04.237 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:04.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:32:04.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:32:04.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:32:04.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:32:04.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:32:04.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:32:04.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:32:04.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:32:04.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:32:04.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:32:04.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:32:04.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:32:04.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:32:04.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-17T12:32:04.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:32:04.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=04e59172-fa9e-4a4b-811a-5c2543d33e33 2026-02-17T12:32:04.239 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 04e59172-fa9e-4a4b-811a-5c2543d33e33 2026-02-17T12:32:04.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 04e59172-fa9e-4a4b-811a-5c2543d33e33' 2026-02-17T12:32:04.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:32:04.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDEX5Rp0wNXDhAAmgohEIu7GvZDKxhtKukSsQ== 2026-02-17T12:32:04.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDEX5Rp0wNXDhAAmgohEIu7GvZDKxhtKukSsQ=="}' 2026-02-17T12:32:04.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 04e59172-fa9e-4a4b-811a-5c2543d33e33 -i td/osd-scrub-test/0/new.json 2026-02-17T12:32:04.335 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:32:04.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-17T12:32:04.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDEX5Rp0wNXDhAAmgohEIu7GvZDKxhtKukSsQ== --osd-uuid 04e59172-fa9e-4a4b-811a-5c2543d33e33 2026-02-17T12:32:04.359 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:04.352+0000 7fcbbdea38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:04.360 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:04.352+0000 7fcbbdea38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:04.361 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:04.352+0000 7fcbbdea38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:04.362 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:04.352+0000 7fcbbdea38c0 -1 bdev(0x555d77133c00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:32:04.362 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:04.352+0000 7fcbbdea38c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T12:32:07.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-17T12:32:07.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:32:07.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:32:07.168 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:32:07.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:32:07.359 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:32:07.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:32:07.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:32:07.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:32:07.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:32:07.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:32:07.380 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:07.372+0000 7fc0e6d978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:07.381 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:07.372+0000 7fc0e6d978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:07.383 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:07.376+0000 7fc0e6d978c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:07.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:32:07.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:32:07.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:32:07.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:32:07.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:32:07.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:07.520 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:32:07.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:32:07.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:07.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:32:07.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:08.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:08.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:08.673 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:32:08.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:32:08.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:08.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:32:08.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:08.888 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:08.880+0000 7fc0e6d978c0 -1 Falling back to public interface 2026-02-17T12:32:09.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:09.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:09.821 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:32:09.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:32:09.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:09.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:32:09.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:10.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:10.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:10.973 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:32:10.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:32:10.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:10.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:32:11.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:11.922 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:11.912+0000 7fc0e6d978c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:32:12.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:12.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:12.126 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:32:12.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:32:12.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:12.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:32:12.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:13.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:13.286 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:13.287 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:32:13.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:32:13.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:13.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:32:13.435 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3846051133,v1:127.0.0.1:6803/3846051133] [v2:127.0.0.1:6804/3846051133,v1:127.0.0.1:6805/3846051133] exists,up 04e59172-fa9e-4a4b-811a-5c2543d33e33 2026-02-17T12:32:13.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:32:13.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:32:13.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:32:13.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:662: TEST_pg_dump_objects_scrubbed: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:32:13.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:664: TEST_pg_dump_objects_scrubbed: run_osd td/osd-scrub-test 1 2026-02-17T12:32:13.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:32:13.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:32:13.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:32:13.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:32:13.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-17T12:32:13.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:32:13.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:32:13.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:32:13.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:32:13.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-17T12:32:13.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-17T12:32:13.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:32:13.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:32:13.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:32:13.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:32:13.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:32:13.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:32:13.440 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:32:13.440 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:13.440 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:13.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:32:13.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:32:13.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:32:13.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:32:13.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:32:13.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:32:13.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:32:13.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:32:13.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:32:13.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:32:13.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:32:13.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:32:13.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:32:13.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-17T12:32:13.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:32:13.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=2d5c79f8-1d6d-4753-b20d-e14726d08ba4 2026-02-17T12:32:13.442 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 2d5c79f8-1d6d-4753-b20d-e14726d08ba4 2026-02-17T12:32:13.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 2d5c79f8-1d6d-4753-b20d-e14726d08ba4' 2026-02-17T12:32:13.443 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:32:13.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDNX5RpzvWeGhAAf86oXv6NKb8/eAoEg/339A== 2026-02-17T12:32:13.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDNX5RpzvWeGhAAf86oXv6NKb8/eAoEg/339A=="}' 2026-02-17T12:32:13.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 2d5c79f8-1d6d-4753-b20d-e14726d08ba4 -i td/osd-scrub-test/1/new.json 2026-02-17T12:32:13.605 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:32:13.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-17T12:32:13.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDNX5RpzvWeGhAAf86oXv6NKb8/eAoEg/339A== --osd-uuid 2d5c79f8-1d6d-4753-b20d-e14726d08ba4 2026-02-17T12:32:13.630 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:13.620+0000 7f7d5f8f48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:13.631 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:13.624+0000 7f7d5f8f48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:13.632 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:13.624+0000 7f7d5f8f48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:13.633 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:13.624+0000 7f7d5f8f48c0 -1 bdev(0x55c543b0dc00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:32:13.633 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:13.624+0000 7f7d5f8f48c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T12:32:15.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-17T12:32:15.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:32:15.916 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:32:15.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:32:15.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:32:16.106 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:32:16.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:32:16.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:32:16.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:32:16.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:32:16.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:32:16.127 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:16.120+0000 7fac66baf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:16.129 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:16.120+0000 7fac66baf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:16.130 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:16.120+0000 7fac66baf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:16.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:32:16.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:32:16.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:32:16.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:32:16.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:32:16.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:16.262 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:32:16.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:32:16.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:16.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:32:16.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:17.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:17.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:17.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:32:17.411 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:32:17.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:17.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:32:17.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:17.608 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:17.600+0000 7fac66baf8c0 -1 Falling back to public interface 2026-02-17T12:32:18.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:18.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:18.559 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:32:18.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:32:18.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:18.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:32:18.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:19.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:19.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:19.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:32:19.709 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:32:19.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:19.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:32:19.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:20.581 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:20.572+0000 7fac66baf8c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:32:20.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:20.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:20.859 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:32:20.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:32:20.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:20.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:32:21.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:22.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:22.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:22.010 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:32:22.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:32:22.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:22.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:32:22.156 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/719891431,v1:127.0.0.1:6811/719891431] [v2:127.0.0.1:6812/719891431,v1:127.0.0.1:6813/719891431] exists,up 2d5c79f8-1d6d-4753-b20d-e14726d08ba4 2026-02-17T12:32:22.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:32:22.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:32:22.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:32:22.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:662: TEST_pg_dump_objects_scrubbed: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:32:22.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:664: TEST_pg_dump_objects_scrubbed: run_osd td/osd-scrub-test 2 2026-02-17T12:32:22.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:32:22.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:32:22.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:32:22.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:32:22.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-17T12:32:22.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:32:22.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:32:22.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:32:22.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:32:22.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-17T12:32:22.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-17T12:32:22.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:32:22.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:32:22.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:32:22.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:32:22.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:32:22.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:32:22.162 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:32:22.162 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:22.162 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:22.162 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:32:22.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:32:22.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:32:22.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:32:22.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:32:22.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:32:22.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:32:22.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:32:22.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:32:22.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:32:22.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:32:22.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:32:22.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:32:22.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-17T12:32:22.163 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:32:22.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=4b4510cd-3ebc-4797-9fd5-54857b6c5584 2026-02-17T12:32:22.164 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 4b4510cd-3ebc-4797-9fd5-54857b6c5584 2026-02-17T12:32:22.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 4b4510cd-3ebc-4797-9fd5-54857b6c5584' 2026-02-17T12:32:22.164 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:32:22.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDWX5RpaeD4CRAAk8wM3UqFicLoYV8T3BRGvg== 2026-02-17T12:32:22.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDWX5RpaeD4CRAAk8wM3UqFicLoYV8T3BRGvg=="}' 2026-02-17T12:32:22.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 4b4510cd-3ebc-4797-9fd5-54857b6c5584 -i td/osd-scrub-test/2/new.json 2026-02-17T12:32:22.324 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:32:22.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-17T12:32:22.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDWX5RpaeD4CRAAk8wM3UqFicLoYV8T3BRGvg== --osd-uuid 4b4510cd-3ebc-4797-9fd5-54857b6c5584 2026-02-17T12:32:22.348 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:22.340+0000 7feccc8a58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:22.349 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:22.340+0000 7feccc8a58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:22.350 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:22.344+0000 7feccc8a58c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:22.351 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:22.344+0000 7feccc8a58c0 -1 bdev(0x558872f29c00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:32:22.351 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:22.344+0000 7feccc8a58c0 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-17T12:32:24.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-17T12:32:24.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:32:24.616 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T12:32:24.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:32:24.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:32:24.807 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T12:32:24.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:32:24.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:32:24.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:32:24.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:32:24.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:32:24.828 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:24.820+0000 7fcb8a0318c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:24.830 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:24.820+0000 7fcb8a0318c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:24.831 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:24.824+0000 7fcb8a0318c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:24.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:32:24.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:32:24.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:32:24.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:32:24.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:32:24.963 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:32:24.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:24.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:32:24.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:24.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:32:25.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:26.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:26.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:26.112 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:32:26.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:32:26.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:26.113 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:32:26.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:26.340 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:26.332+0000 7fcb8a0318c0 -1 Falling back to public interface 2026-02-17T12:32:27.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:27.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:27.262 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:32:27.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:32:27.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:27.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:32:27.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:28.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:28.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:28.412 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:32:28.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:32:28.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:28.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:32:28.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:29.323 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:29.316+0000 7fcb8a0318c0 -1 osd.2 0 log_to_monitors true 2026-02-17T12:32:29.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:29.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:29.564 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:32:29.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:32:29.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:29.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:32:29.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:30.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:30.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:30.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:32:30.719 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:32:30.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:30.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:32:30.864 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/788069453,v1:127.0.0.1:6819/788069453] [v2:127.0.0.1:6820/788069453,v1:127.0.0.1:6821/788069453] exists,up 4b4510cd-3ebc-4797-9fd5-54857b6c5584 2026-02-17T12:32:30.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:32:30.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:32:30.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:32:30.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:668: TEST_pg_dump_objects_scrubbed: create_pool test 1 1 2026-02-17T12:32:30.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T12:32:31.059 INFO:tasks.workunit.client.0.trial128.stderr:pool 'test' created 2026-02-17T12:32:31.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:32:32.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:669: TEST_pg_dump_objects_scrubbed: wait_for_clean 2026-02-17T12:32:32.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:32:32.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:32:32.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:32:32.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:32:32.073 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:32:32.073 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:32:32.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:32:32.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:32:32.073 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:32:32.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:32:32.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:32:32.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:32:32.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:32:32.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:32:32.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:32:32.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:32:32.334 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:32:32.334 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:32:32.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:32:32.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:32:32.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:32:32.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836501 2026-02-17T12:32:32.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836501 2026-02-17T12:32:32.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501' 2026-02-17T12:32:32.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:32:32.404 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:32:32.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705676 2026-02-17T12:32:32.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705676 2026-02-17T12:32:32.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-38654705676' 2026-02-17T12:32:32.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:32:32.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:32:32.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574852 2026-02-17T12:32:32.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574852 2026-02-17T12:32:32.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-38654705676 2-55834574852' 2026-02-17T12:32:32.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:32:32.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836501 2026-02-17T12:32:32.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:32:32.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:32:32.549 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836501 2026-02-17T12:32:32.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:32:32.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836501 2026-02-17T12:32:32.551 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836501 2026-02-17T12:32:32.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836501' 2026-02-17T12:32:32.552 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:32:32.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836501 -lt 21474836501 2026-02-17T12:32:32.706 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:32:32.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705676 2026-02-17T12:32:32.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:32:32.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:32:32.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705676 2026-02-17T12:32:32.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:32:32.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705676 2026-02-17T12:32:32.710 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705676 2026-02-17T12:32:32.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705676' 2026-02-17T12:32:32.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:32:32.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705676 -lt 38654705676 2026-02-17T12:32:32.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:32:32.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574852 2026-02-17T12:32:32.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:32:32.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:32:32.861 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574852 2026-02-17T12:32:32.861 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:32:32.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574852 2026-02-17T12:32:32.863 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574852 2026-02-17T12:32:32.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574852' 2026-02-17T12:32:32.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:32:33.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574851 -lt 55834574852 2026-02-17T12:32:33.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:32:34.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:32:34.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:32:34.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574851 -lt 55834574852 2026-02-17T12:32:34.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:32:35.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-17T12:32:35.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:32:35.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574854 -lt 55834574852 2026-02-17T12:32:35.329 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:32:35.330 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:32:35.330 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:32:35.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:32:35.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:32:35.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:32:35.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:32:35.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:32:35.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:32:35.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:32:35.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:32:35.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:32:35.666 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:32:35.667 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:32:35.667 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:32:35.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:32:35.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:32:35.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:32:35.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:670: TEST_pg_dump_objects_scrubbed: ceph osd dump 2026-02-17T12:32:35.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:670: TEST_pg_dump_objects_scrubbed: grep '^pool.*['\'']test['\'']' 2026-02-17T12:32:35.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:670: TEST_pg_dump_objects_scrubbed: awk '{ print $2 }' 2026-02-17T12:32:36.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:670: TEST_pg_dump_objects_scrubbed: poolid=1 2026-02-17T12:32:36.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:672: TEST_pg_dump_objects_scrubbed: dd if=/dev/urandom of=testdata.508089 bs=1032 count=1 2026-02-17T12:32:36.006 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records in 2026-02-17T12:32:36.006 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records out 2026-02-17T12:32:36.006 INFO:tasks.workunit.client.0.trial128.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.6278e-05 s, 28.4 MB/s 2026-02-17T12:32:36.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: seq 1 15 2026-02-17T12:32:36.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T12:32:36.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj1 testdata.508089 2026-02-17T12:32:36.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T12:32:36.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj2 testdata.508089 2026-02-17T12:32:36.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T12:32:36.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj3 testdata.508089 2026-02-17T12:32:36.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T12:32:36.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj4 testdata.508089 2026-02-17T12:32:36.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T12:32:36.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj5 testdata.508089 2026-02-17T12:32:36.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T12:32:36.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj6 testdata.508089 2026-02-17T12:32:36.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T12:32:36.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj7 testdata.508089 2026-02-17T12:32:36.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T12:32:36.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj8 testdata.508089 2026-02-17T12:32:36.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T12:32:36.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj9 testdata.508089 2026-02-17T12:32:36.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T12:32:36.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj10 testdata.508089 2026-02-17T12:32:36.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T12:32:36.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj11 testdata.508089 2026-02-17T12:32:36.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T12:32:36.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj12 testdata.508089 2026-02-17T12:32:36.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T12:32:36.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj13 testdata.508089 2026-02-17T12:32:36.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T12:32:36.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj14 testdata.508089 2026-02-17T12:32:36.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:673: TEST_pg_dump_objects_scrubbed: for i in `seq 1 $objects` 2026-02-17T12:32:36.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:675: TEST_pg_dump_objects_scrubbed: rados -p test put obj15 testdata.508089 2026-02-17T12:32:36.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:677: TEST_pg_dump_objects_scrubbed: rm -f testdata.508089 2026-02-17T12:32:36.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:679: TEST_pg_dump_objects_scrubbed: local pgid=1.0 2026-02-17T12:32:36.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:681: TEST_pg_dump_objects_scrubbed: pg_scrub 1.0 2026-02-17T12:32:36.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1932: pg_scrub: local pgid=1.0 2026-02-17T12:32:36.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1934: pg_scrub: wait_for_pg_clean 1.0 2026-02-17T12:32:36.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=1.0 2026-02-17T12:32:36.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T12:32:36.344 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:32:36.344 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:32:36.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:32:36.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:32:36.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:32:36.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T12:32:36.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T12:32:36.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T12:32:36.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T12:32:36.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:32:36.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:32:36.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:32:36.775 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:32:36.775 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:32:36.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:32:36.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:32:36.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:32:36.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836506 2026-02-17T12:32:36.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836506 2026-02-17T12:32:36.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836506' 2026-02-17T12:32:36.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:32:36.843 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:32:36.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705682 2026-02-17T12:32:36.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705682 2026-02-17T12:32:36.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836506 1-38654705682' 2026-02-17T12:32:36.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:32:36.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:32:36.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574857 2026-02-17T12:32:36.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574857 2026-02-17T12:32:36.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836506 1-38654705682 2-55834574857' 2026-02-17T12:32:36.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:32:36.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836506 2026-02-17T12:32:36.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:32:36.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:32:36.987 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836506 2026-02-17T12:32:36.987 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:32:36.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836506 2026-02-17T12:32:36.989 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836506 2026-02-17T12:32:36.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836506' 2026-02-17T12:32:36.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:32:37.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836505 -lt 21474836506 2026-02-17T12:32:37.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:32:38.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:32:38.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:32:38.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836505 -lt 21474836506 2026-02-17T12:32:38.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:32:39.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-17T12:32:39.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:32:39.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836508 -lt 21474836506 2026-02-17T12:32:39.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:32:39.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705682 2026-02-17T12:32:39.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:32:39.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:32:39.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705682 2026-02-17T12:32:39.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:32:39.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705682 2026-02-17T12:32:39.455 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705682 2026-02-17T12:32:39.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705682' 2026-02-17T12:32:39.455 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:32:39.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705683 -lt 38654705682 2026-02-17T12:32:39.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:32:39.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574857 2026-02-17T12:32:39.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:32:39.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:32:39.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574857 2026-02-17T12:32:39.615 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:32:39.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574857 2026-02-17T12:32:39.617 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 55834574857 2026-02-17T12:32:39.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574857' 2026-02-17T12:32:39.617 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:32:39.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574859 -lt 55834574857 2026-02-17T12:32:39.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T12:32:39.778 INFO:tasks.workunit.client.0.trial128.stdout:#---------- 1.0 loop 0 2026-02-17T12:32:39.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T12:32:39.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T12:32:39.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=1.0 2026-02-17T12:32:39.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T12:32:39.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 1.0 query 2026-02-17T12:32:39.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T12:32:39.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T12:32:39.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T12:32:39.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T12:32:39.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T12:32:39.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: get_last_scrub_stamp 1.0 2026-02-17T12:32:39.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:32:39.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:32:39.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:32:39.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:32:40.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1935: pg_scrub: local last_scrub=2026-02-17T12:32:31.054643+0000 2026-02-17T12:32:40.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1936: pg_scrub: ceph pg scrub 1.0 2026-02-17T12:32:40.142 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 1.0 on osd.1 to scrub 2026-02-17T12:32:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1937: pg_scrub: wait_for_scrub 1.0 2026-02-17T12:32:31.054643+0000 2026-02-17T12:32:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T12:32:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:32:31.054643+0000 2026-02-17T12:32:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:32:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:32:40.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:32:40.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:32:40.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:32:40.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:32:40.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:32:40.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:32:40.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:32:31.054643+0000 '>' 2026-02-17T12:32:31.054643+0000 2026-02-17T12:32:40.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:32:41.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:32:41.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:32:41.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:32:41.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:32:41.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:32:41.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:32:41.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:32:41.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:32:31.054643+0000 '>' 2026-02-17T12:32:31.054643+0000 2026-02-17T12:32:41.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:32:42.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:32:42.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:32:42.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:32:42.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:32:42.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:32:42.466 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:32:42.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:32:42.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:32:40.623040+0000 '>' 2026-02-17T12:32:31.054643+0000 2026-02-17T12:32:42.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:32:42.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:682: TEST_pg_dump_objects_scrubbed: ceph pg 1.0 query 2026-02-17T12:32:42.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:682: TEST_pg_dump_objects_scrubbed: jq .info.stats.objects_scrubbed 2026-02-17T12:32:42.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:682: TEST_pg_dump_objects_scrubbed: test 15 = 15 2026-02-17T12:32:42.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:683: TEST_pg_dump_objects_scrubbed: perf_counters td/osd-scrub-test 3 2026-02-17T12:32:42.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-17T12:32:42.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-17T12:32:42.692 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-17T12:32:42.693 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-17T12:32:42.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:32:42.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-17T12:32:42.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:32:42.766 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:32:42.766 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:32:42.766 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:42.766 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:42.767 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:32:42.767 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:32:42.767 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.767 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:42.767 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:42.767 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:42.767 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:42.767 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:42.767 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.767 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.767 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.768 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.768 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:42.768 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:42.768 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.768 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.768 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.768 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.768 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:42.768 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:42.768 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:42.768 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:42.768 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:42.769 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:42.769 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:42.769 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.769 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.769 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.769 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.769 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:42.769 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:42.769 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:42.769 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:42.769 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.769 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.769 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.770 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.770 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:42.770 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.770 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.770 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:42.770 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:32:42.770 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:42.770 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:42.770 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:32:42.770 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:32:42.770 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.770 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:42.771 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:42.771 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:42.771 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:42.771 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:42.771 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.771 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.771 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.771 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.771 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:42.771 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:42.771 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.771 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.771 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.772 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.772 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:42.772 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:42.772 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:42.772 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:42.772 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:42.772 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:42.772 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:42.772 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.772 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.772 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.772 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.773 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:42.773 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:42.773 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:42.773 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:42.773 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.773 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.773 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.773 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.773 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:42.773 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.773 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.773 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:42.774 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:32:42.774 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:42.774 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:42.774 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:32:42.774 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:32:42.774 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.774 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:42.774 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:42.774 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:42.774 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:42.774 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:42.774 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.774 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.775 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.775 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.775 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:42.775 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:42.775 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.775 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.775 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.775 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.775 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:42.775 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:42.775 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:42.775 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:42.775 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:42.776 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:42.776 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:42.776 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.776 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.776 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.776 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.776 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:42.776 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:42.776 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:42.776 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:42.776 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.776 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.776 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.777 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.777 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:42.777 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.777 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.777 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:42.777 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:32:42.777 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:42.777 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:42.777 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:32:42.777 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:32:42.777 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.777 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:42.778 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:42.778 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:42.778 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:42.778 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:42.778 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.778 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.778 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.778 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.778 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:42.778 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:42.778 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.778 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.779 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.779 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.779 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:42.779 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:42.779 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:42.779 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:42.779 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:42.779 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:42.779 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:42.780 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.780 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.780 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.780 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.780 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:42.780 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:42.780 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:42.780 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:42.781 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.781 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.781 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.781 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.781 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:42.781 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.781 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.781 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:32:42.781 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:32:42.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:32:42.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-17T12:32:42.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:32:42.834 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:32:42.834 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:32:42.834 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:42.834 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:42.834 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:32:42.834 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:32:42.834 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.834 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:42.834 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:42.834 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:42.834 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:42.835 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:42.835 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.835 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.835 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.835 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.835 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:42.835 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:42.835 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.835 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.835 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.835 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.835 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:42.836 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:42.836 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:42.836 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:42.836 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:42.836 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:42.836 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:42.836 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.836 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.836 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.836 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.836 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:42.836 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:42.836 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:42.837 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:42.837 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.837 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.837 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.837 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.837 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:42.837 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.837 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.837 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:42.837 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:32:42.837 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:42.837 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:42.838 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:32:42.838 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:32:42.838 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.838 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:42.838 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:42.838 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:42.838 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:42.838 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:42.838 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.838 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.838 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.838 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.838 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:42.839 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:42.839 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.839 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.839 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.839 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.839 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:42.839 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:42.839 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:42.839 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:42.839 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:42.839 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:42.839 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:42.839 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.840 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.840 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.840 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.840 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:42.840 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:42.840 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:42.840 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:42.840 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.840 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.840 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.840 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.840 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:42.841 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.841 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.841 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:42.841 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:32:42.841 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:42.841 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:42.841 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:32:42.841 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:32:42.841 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.841 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:42.841 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:42.841 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:42.841 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:42.842 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:42.842 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.842 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.842 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.842 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.842 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:42.842 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:42.842 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.842 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.842 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.842 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.842 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:42.842 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:42.843 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:42.843 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:42.843 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:42.843 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:42.843 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:42.843 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.843 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.843 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.843 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.843 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:42.843 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:42.843 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:42.843 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:42.844 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.844 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.844 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.844 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.844 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:42.844 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.844 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.844 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:42.844 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:32:42.844 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:42.844 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:42.844 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:32:42.845 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:32:42.845 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.845 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:42.845 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 1, 2026-02-17T12:32:42.845 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 1, 2026-02-17T12:32:42.845 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 1, 2026-02-17T12:32:42.845 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:42.845 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 1, 2026-02-17T12:32:42.845 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.845 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.845 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.845 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:42.845 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:42.846 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.846 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.846 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.846 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.846 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:42.846 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 1, 2026-02-17T12:32:42.846 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:42.846 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:42.846 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:42.846 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:42.846 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:42.847 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.847 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.847 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.847 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.847 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:42.847 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:42.847 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 1, 2026-02-17T12:32:42.847 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:42.848 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.848 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.848 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.848 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.848 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 2 2026-02-17T12:32:42.848 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.848 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.848 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:32:42.848 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:32:42.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:32:42.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-17T12:32:42.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:32:42.906 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:32:42.906 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:32:42.906 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:42.906 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:42.906 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:32:42.906 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:32:42.906 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.906 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:42.907 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:42.907 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:42.907 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:42.907 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:42.907 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.907 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.907 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.907 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.907 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:42.907 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:42.907 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.907 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.908 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.908 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.908 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:42.908 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:42.908 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:42.908 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:42.908 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:42.908 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:42.908 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:42.908 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.909 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.909 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.909 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.909 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:42.909 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:42.909 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:42.909 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:42.909 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.909 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.909 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.909 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.909 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:42.910 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.910 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.910 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:42.910 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:32:42.910 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:42.910 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:42.910 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:32:42.910 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:32:42.910 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.910 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:42.910 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:42.910 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:42.911 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:42.911 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:42.911 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.911 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.911 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.911 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.911 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:42.911 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:42.911 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.911 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.911 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.911 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.911 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:42.911 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:42.912 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:42.912 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:42.912 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:42.912 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:42.912 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:42.912 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.912 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.912 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.912 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.912 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:42.912 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:42.913 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:42.913 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:42.913 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.913 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.913 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.913 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.913 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:42.913 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.913 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.913 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:42.913 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:32:42.914 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:42.914 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:42.914 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:32:42.914 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:32:42.914 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.914 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:42.914 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:42.914 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:42.914 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:42.914 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:42.914 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.914 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.914 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.915 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.915 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:42.915 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:42.915 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.915 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.915 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.915 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.915 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:42.915 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:42.915 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:42.915 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:42.915 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:42.915 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:42.916 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:42.916 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.916 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.916 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.916 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.916 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:42.916 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:42.916 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:42.916 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:42.916 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.916 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.916 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.916 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.916 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:42.917 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.917 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.917 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:32:42.917 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:32:42.917 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:32:42.917 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:32:42.917 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:32:42.917 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:32:42.917 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.917 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:32:42.917 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:32:42.917 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:32:42.917 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:32:42.918 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:32:42.918 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.918 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.918 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.918 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.918 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:32:42.918 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:32:42.918 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.918 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.918 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.918 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.918 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:32:42.918 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:32:42.919 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:32:42.919 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:32:42.919 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:32:42.919 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:32:42.919 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:32:42.919 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.920 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.920 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.920 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.920 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:32:42.920 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:32:42.920 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:32:42.920 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:32:42.920 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:32:42.920 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:32:42.920 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:32:42.920 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:32:42.921 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:32:42.921 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.921 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:32:42.921 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:32:42.921 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:32:42.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:685: TEST_pg_dump_objects_scrubbed: teardown td/osd-scrub-test 2026-02-17T12:32:42.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:32:42.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:32:42.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:32:42.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:32:42.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:32:42.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:32:42.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:32:42.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:32:43.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:32:43.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:32:43.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:32:43.025 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:32:43.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:32:43.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:32:43.027 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:32:43.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:32:43.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:32:43.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:32:43.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:32:43.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:32:43.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:32:43.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:32:43.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:32:43.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:43.042 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:43.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:32:43.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:32:43.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:32:43.043 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Teardown Test TEST_pg_dump_objects_scrubbed ------------------ 2026-02-17T12:32:43.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_pg_dump_objects_scrubbed ------------------' 2026-02-17T12:32:43.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T12:32:43.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:32:43.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:32:43.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:32:43.045 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:32:43.045 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:32:43.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:32:43.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:32:43.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:32:43.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:32:43.047 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:32:43.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:32:43.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:32:43.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:32:43.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:32:43.049 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:32:43.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:32:43.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:32:43.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:32:43.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:32:43.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:32:43.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:32:43.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:32:43.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:32:43.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:43.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:43.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:32:43.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:32:43.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:32:43.056 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Complete Test TEST_pg_dump_objects_scrubbed ------------------ 2026-02-17T12:32:43.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_pg_dump_objects_scrubbed ------------------' 2026-02-17T12:32:43.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T12:32:43.057 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Prepare Test TEST_scrub_abort ------------------- 2026-02-17T12:32:43.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_scrub_abort -------------------' 2026-02-17T12:32:43.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T12:32:43.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T12:32:43.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T12:32:43.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:32:43.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:32:43.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:32:43.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:32:43.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:32:43.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:32:43.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:32:43.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:32:43.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:32:43.060 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:32:43.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:32:43.061 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:32:43.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:32:43.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:32:43.062 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:32:43.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:32:43.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:32:43.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:32:43.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:32:43.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:32:43.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:32:43.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:32:43.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:32:43.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:43.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:43.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:32:43.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:32:43.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:32:43.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T12:32:43.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:32:43.070 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:43.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:43.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.508089 2026-02-17T12:32:43.072 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:32:43.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:32:43.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:32:43.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T12:32:43.073 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Run Test TEST_scrub_abort ----------------------- 2026-02-17T12:32:43.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_scrub_abort -----------------------' 2026-02-17T12:32:43.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_scrub_abort td/osd-scrub-test 2026-02-17T12:32:43.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:409: TEST_scrub_abort: local dir=td/osd-scrub-test 2026-02-17T12:32:43.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:410: TEST_scrub_abort: _scrub_abort td/osd-scrub-test scrub 2026-02-17T12:32:43.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:298: _scrub_abort: local dir=td/osd-scrub-test 2026-02-17T12:32:43.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:299: _scrub_abort: local poolname=test 2026-02-17T12:32:43.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:300: _scrub_abort: local OSDS=3 2026-02-17T12:32:43.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:301: _scrub_abort: local objects=1000 2026-02-17T12:32:43.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:302: _scrub_abort: local type=scrub 2026-02-17T12:32:43.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:304: _scrub_abort: TESTDATA=testdata.508089 2026-02-17T12:32:43.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:305: _scrub_abort: test scrub = scrub 2026-02-17T12:32:43.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:307: _scrub_abort: stopscrub=noscrub 2026-02-17T12:32:43.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:308: _scrub_abort: check=noscrub 2026-02-17T12:32:43.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:314: _scrub_abort: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-17T12:32:43.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-test 2026-02-17T12:32:43.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:32:43.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:32:43.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:32:43.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-test/a 2026-02-17T12:32:43.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-17T12:32:43.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:32:43.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:32:43.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:32:43.094 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:32:43.094 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:43.094 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:43.095 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:32:43.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-17T12:32:43.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:32:43.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:32:43.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:32:43.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:32:43.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:32:43.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:32:43.124 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:32:43.124 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:32:43.124 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:32:43.124 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:32:43.124 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:43.124 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:43.124 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-mon.a.asok 2026-02-17T12:32:43.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:32:43.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.508089/ceph-mon.a.asok config get fsid 2026-02-17T12:32:43.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:32:43.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:32:43.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:32:43.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:32:43.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:32:43.180 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:32:43.180 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:32:43.181 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:32:43.181 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:32:43.181 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:43.181 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:43.181 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-mon.a.asok 2026-02-17T12:32:43.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:32:43.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.508089/ceph-mon.a.asok config get mon_host 2026-02-17T12:32:43.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:315: _scrub_abort: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-17T12:32:43.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-test 2026-02-17T12:32:43.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:32:43.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:32:43.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:32:43.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-test/x 2026-02-17T12:32:43.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:32:43.326 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:32:43.326 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:32:43.326 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:32:43.327 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:32:43.327 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:43.327 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:43.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:32:43.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:32:43.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T12:32:43.348 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: expr 3 - 1 2026-02-17T12:32:43.349 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: seq 0 2 2026-02-17T12:32:43.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:32:43.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:322: _scrub_abort: run_osd td/osd-scrub-test 0 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T12:32:43.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:32:43.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:32:43.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:32:43.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:32:43.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-17T12:32:43.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:32:43.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:32:43.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:32:43.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:32:43.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-17T12:32:43.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-17T12:32:43.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:32:43.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:32:43.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:32:43.351 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:32:43.351 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:32:43.351 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-17T12:32:43.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-17T12:32:43.353 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:32:43.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=f786cd11-49c7-4ce1-9b32-25f879aa2e4c 2026-02-17T12:32:43.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 f786cd11-49c7-4ce1-9b32-25f879aa2e4c' 2026-02-17T12:32:43.354 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 f786cd11-49c7-4ce1-9b32-25f879aa2e4c 2026-02-17T12:32:43.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:32:43.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDrX5RpreZ+FRAAnpy82fJdGrtmKPo4TeoNJA== 2026-02-17T12:32:43.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDrX5RpreZ+FRAAnpy82fJdGrtmKPo4TeoNJA=="}' 2026-02-17T12:32:43.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new f786cd11-49c7-4ce1-9b32-25f879aa2e4c -i td/osd-scrub-test/0/new.json 2026-02-17T12:32:43.456 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:32:43.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-17T12:32:43.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQDrX5RpreZ+FRAAnpy82fJdGrtmKPo4TeoNJA== --osd-uuid f786cd11-49c7-4ce1-9b32-25f879aa2e4c 2026-02-17T12:32:43.481 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:43.471+0000 7f4adf73d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:43.483 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:43.475+0000 7f4adf73d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:43.483 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:43.475+0000 7f4adf73d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:43.484 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:43.475+0000 7f4adf73d8c0 -1 bdev(0x561adacddc00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:32:43.484 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:43.475+0000 7f4adf73d8c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T12:32:46.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-17T12:32:46.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:32:46.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:32:46.265 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:32:46.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:32:46.455 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:32:46.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:32:46.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T12:32:46.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:32:46.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:32:46.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:32:46.479 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:46.471+0000 7fd02b8128c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:46.481 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:46.471+0000 7fd02b8128c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:46.482 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:46.475+0000 7fd02b8128c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:46.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:32:46.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:32:46.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:32:46.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:32:46.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:32:46.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:46.611 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:32:46.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:32:46.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:46.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:32:46.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:47.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:47.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:47.761 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:32:47.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:32:47.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:47.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:32:47.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:47.988 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:47.979+0000 7fd02b8128c0 -1 Falling back to public interface 2026-02-17T12:32:48.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:48.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:48.914 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:32:48.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:32:48.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:48.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:32:49.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:50.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:50.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:50.062 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:32:50.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:32:50.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:50.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:32:50.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:51.006 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:50.995+0000 7fd02b8128c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:32:51.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:51.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:51.223 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:32:51.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:32:51.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:51.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:32:51.369 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/53504729,v1:127.0.0.1:6803/53504729] [v2:127.0.0.1:6804/53504729,v1:127.0.0.1:6805/53504729] exists,up f786cd11-49c7-4ce1-9b32-25f879aa2e4c 2026-02-17T12:32:51.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:32:51.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:32:51.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:32:51.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:32:51.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:322: _scrub_abort: run_osd td/osd-scrub-test 1 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T12:32:51.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:32:51.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:32:51.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:32:51.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:32:51.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-17T12:32:51.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:32:51.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:32:51.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:32:51.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:32:51.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-17T12:32:51.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-17T12:32:51.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:32:51.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:32:51.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:32:51.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:32:51.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:32:51.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:32:51.375 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:32:51.375 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:51.375 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:51.375 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:32:51.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:32:51.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:32:51.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:32:51.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:32:51.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:32:51.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:32:51.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:32:51.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:32:51.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:32:51.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:32:51.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:32:51.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-17T12:32:51.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-17T12:32:51.376 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:32:51.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b2c73621-df3e-481e-bf56-e416ac9a5e5e 2026-02-17T12:32:51.377 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 b2c73621-df3e-481e-bf56-e416ac9a5e5e 2026-02-17T12:32:51.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 b2c73621-df3e-481e-bf56-e416ac9a5e5e' 2026-02-17T12:32:51.378 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:32:51.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDzX5RpkfaeFhAALzvspz9OpUOh/v2FVfvE3g== 2026-02-17T12:32:51.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDzX5RpkfaeFhAALzvspz9OpUOh/v2FVfvE3g=="}' 2026-02-17T12:32:51.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b2c73621-df3e-481e-bf56-e416ac9a5e5e -i td/osd-scrub-test/1/new.json 2026-02-17T12:32:51.544 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:32:51.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-17T12:32:51.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQDzX5RpkfaeFhAALzvspz9OpUOh/v2FVfvE3g== --osd-uuid b2c73621-df3e-481e-bf56-e416ac9a5e5e 2026-02-17T12:32:51.580 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:51.571+0000 7f6925ba68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:51.581 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:51.571+0000 7f6925ba68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:51.583 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:51.575+0000 7f6925ba68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:51.583 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:51.575+0000 7f6925ba68c0 -1 bdev(0x561586abbc00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:32:51.584 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:51.575+0000 7f6925ba68c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T12:32:53.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-17T12:32:53.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:32:53.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:32:53.873 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:32:53.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:32:54.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:32:54.063 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:32:54.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T12:32:54.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:32:54.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:32:54.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:32:54.086 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:54.075+0000 7f2a280038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:54.088 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:54.079+0000 7f2a280038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:54.089 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:54.079+0000 7f2a280038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:54.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:32:54.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:32:54.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:32:54.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:32:54.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:32:54.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:54.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:32:54.219 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:32:54.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:54.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:32:54.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:55.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:55.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:55.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:32:55.367 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:32:55.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:55.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:32:55.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:55.592 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:55.583+0000 7f2a280038c0 -1 Falling back to public interface 2026-02-17T12:32:56.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:56.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:56.517 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:32:56.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:32:56.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:56.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:32:56.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:57.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:57.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:57.665 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:32:57.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:32:57.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:57.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:32:57.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:32:58.545 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:58.535+0000 7f2a280038c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:32:58.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:32:58.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:32:58.816 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:32:58.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:32:58.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:32:58.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:32:58.963 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/859801775,v1:127.0.0.1:6811/859801775] [v2:127.0.0.1:6812/859801775,v1:127.0.0.1:6813/859801775] exists,up b2c73621-df3e-481e-bf56-e416ac9a5e5e 2026-02-17T12:32:58.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:32:58.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:32:58.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:32:58.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:316: _scrub_abort: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:32:58.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:322: _scrub_abort: run_osd td/osd-scrub-test 2 --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T12:32:58.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:32:58.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:32:58.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:32:58.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:32:58.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:32:58.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:32:58.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:32:58.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:32:58.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:32:58.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:32:58.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:32:58.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:32:58.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq' 2026-02-17T12:32:58.966 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-17T12:32:58.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:32:58.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=7401dd28-5ee3-4eb7-999e-3699f662d941 2026-02-17T12:32:58.967 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 7401dd28-5ee3-4eb7-999e-3699f662d941 2026-02-17T12:32:58.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 7401dd28-5ee3-4eb7-999e-3699f662d941' 2026-02-17T12:32:58.967 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:32:58.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQD6X5RpyMwVOhAAgaFVC93BotbnWy5zkWTbBA== 2026-02-17T12:32:58.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQD6X5RpyMwVOhAAgaFVC93BotbnWy5zkWTbBA=="}' 2026-02-17T12:32:58.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 7401dd28-5ee3-4eb7-999e-3699f662d941 -i td/osd-scrub-test/2/new.json 2026-02-17T12:32:59.132 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:32:59.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-17T12:32:59.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq --mkfs --key AQD6X5RpyMwVOhAAgaFVC93BotbnWy5zkWTbBA== --osd-uuid 7401dd28-5ee3-4eb7-999e-3699f662d941 2026-02-17T12:32:59.159 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:59.151+0000 7f8031ccf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:59.160 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:59.151+0000 7f8031ccf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:59.162 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:59.151+0000 7f8031ccf8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:32:59.162 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:59.155+0000 7f8031ccf8c0 -1 bdev(0x5584dd811c00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:32:59.162 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:32:59.155+0000 7f8031ccf8c0 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-17T12:33:01.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-17T12:33:01.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:33:01.452 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T12:33:01.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:33:01.452 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:33:01.640 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T12:33:01.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:33:01.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_pool_default_pg_autoscale_mode=off --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_sleep=5.0 --osd_scrub_interval_randomize_ratio=0 --osd_op_queue=wpq 2026-02-17T12:33:01.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:33:01.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:33:01.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:33:01.664 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:33:01.655+0000 7fd81d8008c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:33:01.665 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:33:01.655+0000 7fd81d8008c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:33:01.667 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:33:01.659+0000 7fd81d8008c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:33:01.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:33:01.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:33:01.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:33:01.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:33:01.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:33:01.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:33:01.798 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:33:01.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:33:01.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:33:01.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:33:01.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:33:02.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:33:02.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:33:02.946 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:33:02.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:33:02.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:33:02.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:33:03.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:33:03.156 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:33:03.147+0000 7fd81d8008c0 -1 Falling back to public interface 2026-02-17T12:33:04.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:33:04.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:33:04.097 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:33:04.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:33:04.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:33:04.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:33:04.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:33:05.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:33:05.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:33:05.247 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:33:05.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:33:05.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:33:05.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:33:05.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:33:06.147 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:33:06.139+0000 7fd81d8008c0 -1 osd.2 0 log_to_monitors true 2026-02-17T12:33:06.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:33:06.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:33:06.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:33:06.395 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:33:06.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:33:06.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:33:06.541 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/345689100,v1:127.0.0.1:6819/345689100] [v2:127.0.0.1:6820/345689100,v1:127.0.0.1:6821/345689100] exists,up 7401dd28-5ee3-4eb7-999e-3699f662d941 2026-02-17T12:33:06.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:33:06.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:33:06.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:33:06.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:330: _scrub_abort: create_pool test 1 1 2026-02-17T12:33:06.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T12:33:06.709 INFO:tasks.workunit.client.0.trial128.stderr:pool 'test' created 2026-02-17T12:33:06.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:33:07.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:331: _scrub_abort: wait_for_clean 2026-02-17T12:33:07.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:33:07.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:33:07.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:33:07.720 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:33:07.720 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:33:07.720 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:33:07.720 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:33:07.720 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:33:07.720 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:33:07.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:33:07.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:33:07.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:33:07.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:33:07.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:33:07.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:33:07.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:33:07.976 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:33:07.976 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:33:07.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:33:07.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:07.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:33:08.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836497 2026-02-17T12:33:08.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836497 2026-02-17T12:33:08.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497' 2026-02-17T12:33:08.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:08.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:33:08.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672970 2026-02-17T12:33:08.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672970 2026-02-17T12:33:08.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-42949672970' 2026-02-17T12:33:08.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:08.120 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:33:08.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542146 2026-02-17T12:33:08.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542146 2026-02-17T12:33:08.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836497 1-42949672970 2-60129542146' 2026-02-17T12:33:08.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:08.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836497 2026-02-17T12:33:08.193 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:08.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:33:08.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836497 2026-02-17T12:33:08.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:08.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836497 2026-02-17T12:33:08.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2026-02-17T12:33:08.197 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836497 2026-02-17T12:33:08.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:08.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836497 2026-02-17T12:33:08.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:33:09.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:33:09.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:09.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836497 2026-02-17T12:33:09.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:33:10.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-17T12:33:10.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:10.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836497 2026-02-17T12:33:10.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:10.667 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672970 2026-02-17T12:33:10.667 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:10.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:33:10.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672970 2026-02-17T12:33:10.669 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:10.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672970 2026-02-17T12:33:10.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-17T12:33:10.671 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672970 2026-02-17T12:33:10.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:33:10.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672971 -lt 42949672970 2026-02-17T12:33:10.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:10.820 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542146 2026-02-17T12:33:10.821 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:10.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:33:10.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542146 2026-02-17T12:33:10.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:10.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542146 2026-02-17T12:33:10.824 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542146 2026-02-17T12:33:10.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-17T12:33:10.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:33:10.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542148 -lt 60129542146 2026-02-17T12:33:10.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:33:10.990 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:33:10.990 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:33:11.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:33:11.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:33:11.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:33:11.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:33:11.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:33:11.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:33:11.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:33:11.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:33:11.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:33:11.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:33:11.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:33:11.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:33:11.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:33:11.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:33:11.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:33:11.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: ceph osd dump 2026-02-17T12:33:11.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: grep '^pool.*['\'']test['\'']' 2026-02-17T12:33:11.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: awk '{ print $2 }' 2026-02-17T12:33:11.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:332: _scrub_abort: poolid=1 2026-02-17T12:33:11.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:334: _scrub_abort: dd if=/dev/urandom of=testdata.508089 bs=1032 count=1 2026-02-17T12:33:11.664 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records in 2026-02-17T12:33:11.664 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records out 2026-02-17T12:33:11.664 INFO:tasks.workunit.client.0.trial128.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 5.343e-05 s, 19.3 MB/s 2026-02-17T12:33:11.665 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: seq 1 1000 2026-02-17T12:33:11.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:11.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj1 testdata.508089 2026-02-17T12:33:11.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:11.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj2 testdata.508089 2026-02-17T12:33:11.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:11.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj3 testdata.508089 2026-02-17T12:33:11.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:11.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj4 testdata.508089 2026-02-17T12:33:11.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:11.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj5 testdata.508089 2026-02-17T12:33:11.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:11.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj6 testdata.508089 2026-02-17T12:33:11.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:11.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj7 testdata.508089 2026-02-17T12:33:11.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:11.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj8 testdata.508089 2026-02-17T12:33:11.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:11.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj9 testdata.508089 2026-02-17T12:33:11.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:11.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj10 testdata.508089 2026-02-17T12:33:11.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:11.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj11 testdata.508089 2026-02-17T12:33:11.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:11.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj12 testdata.508089 2026-02-17T12:33:11.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:11.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj13 testdata.508089 2026-02-17T12:33:11.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:11.973 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj14 testdata.508089 2026-02-17T12:33:11.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:11.998 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj15 testdata.508089 2026-02-17T12:33:12.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj16 testdata.508089 2026-02-17T12:33:12.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj17 testdata.508089 2026-02-17T12:33:12.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj18 testdata.508089 2026-02-17T12:33:12.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj19 testdata.508089 2026-02-17T12:33:12.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj20 testdata.508089 2026-02-17T12:33:12.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj21 testdata.508089 2026-02-17T12:33:12.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj22 testdata.508089 2026-02-17T12:33:12.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj23 testdata.508089 2026-02-17T12:33:12.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj24 testdata.508089 2026-02-17T12:33:12.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj25 testdata.508089 2026-02-17T12:33:12.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj26 testdata.508089 2026-02-17T12:33:12.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj27 testdata.508089 2026-02-17T12:33:12.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj28 testdata.508089 2026-02-17T12:33:12.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj29 testdata.508089 2026-02-17T12:33:12.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj30 testdata.508089 2026-02-17T12:33:12.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj31 testdata.508089 2026-02-17T12:33:12.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj32 testdata.508089 2026-02-17T12:33:12.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj33 testdata.508089 2026-02-17T12:33:12.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj34 testdata.508089 2026-02-17T12:33:12.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj35 testdata.508089 2026-02-17T12:33:12.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj36 testdata.508089 2026-02-17T12:33:12.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj37 testdata.508089 2026-02-17T12:33:12.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj38 testdata.508089 2026-02-17T12:33:12.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj39 testdata.508089 2026-02-17T12:33:12.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj40 testdata.508089 2026-02-17T12:33:12.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj41 testdata.508089 2026-02-17T12:33:12.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.614 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj42 testdata.508089 2026-02-17T12:33:12.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj43 testdata.508089 2026-02-17T12:33:12.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj44 testdata.508089 2026-02-17T12:33:12.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj45 testdata.508089 2026-02-17T12:33:12.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj46 testdata.508089 2026-02-17T12:33:12.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj47 testdata.508089 2026-02-17T12:33:12.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj48 testdata.508089 2026-02-17T12:33:12.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj49 testdata.508089 2026-02-17T12:33:12.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj50 testdata.508089 2026-02-17T12:33:12.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj51 testdata.508089 2026-02-17T12:33:12.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj52 testdata.508089 2026-02-17T12:33:12.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj53 testdata.508089 2026-02-17T12:33:12.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj54 testdata.508089 2026-02-17T12:33:12.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj55 testdata.508089 2026-02-17T12:33:12.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj56 testdata.508089 2026-02-17T12:33:12.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:12.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj57 testdata.508089 2026-02-17T12:33:13.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj58 testdata.508089 2026-02-17T12:33:13.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj59 testdata.508089 2026-02-17T12:33:13.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj60 testdata.508089 2026-02-17T12:33:13.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj61 testdata.508089 2026-02-17T12:33:13.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj62 testdata.508089 2026-02-17T12:33:13.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.126 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj63 testdata.508089 2026-02-17T12:33:13.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj64 testdata.508089 2026-02-17T12:33:13.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj65 testdata.508089 2026-02-17T12:33:13.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj66 testdata.508089 2026-02-17T12:33:13.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj67 testdata.508089 2026-02-17T12:33:13.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj68 testdata.508089 2026-02-17T12:33:13.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj69 testdata.508089 2026-02-17T12:33:13.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj70 testdata.508089 2026-02-17T12:33:13.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj71 testdata.508089 2026-02-17T12:33:13.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.345 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj72 testdata.508089 2026-02-17T12:33:13.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj73 testdata.508089 2026-02-17T12:33:13.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj74 testdata.508089 2026-02-17T12:33:13.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj75 testdata.508089 2026-02-17T12:33:13.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.427 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj76 testdata.508089 2026-02-17T12:33:13.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj77 testdata.508089 2026-02-17T12:33:13.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj78 testdata.508089 2026-02-17T12:33:13.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj79 testdata.508089 2026-02-17T12:33:13.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.525 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj80 testdata.508089 2026-02-17T12:33:13.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj81 testdata.508089 2026-02-17T12:33:13.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj82 testdata.508089 2026-02-17T12:33:13.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj83 testdata.508089 2026-02-17T12:33:13.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj84 testdata.508089 2026-02-17T12:33:13.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj85 testdata.508089 2026-02-17T12:33:13.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj86 testdata.508089 2026-02-17T12:33:13.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj87 testdata.508089 2026-02-17T12:33:13.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj88 testdata.508089 2026-02-17T12:33:13.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj89 testdata.508089 2026-02-17T12:33:13.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj90 testdata.508089 2026-02-17T12:33:13.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj91 testdata.508089 2026-02-17T12:33:13.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj92 testdata.508089 2026-02-17T12:33:13.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj93 testdata.508089 2026-02-17T12:33:13.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj94 testdata.508089 2026-02-17T12:33:13.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj95 testdata.508089 2026-02-17T12:33:13.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj96 testdata.508089 2026-02-17T12:33:13.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj97 testdata.508089 2026-02-17T12:33:13.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj98 testdata.508089 2026-02-17T12:33:13.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj99 testdata.508089 2026-02-17T12:33:13.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:13.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj100 testdata.508089 2026-02-17T12:33:14.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj101 testdata.508089 2026-02-17T12:33:14.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj102 testdata.508089 2026-02-17T12:33:14.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj103 testdata.508089 2026-02-17T12:33:14.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj104 testdata.508089 2026-02-17T12:33:14.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.115 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj105 testdata.508089 2026-02-17T12:33:14.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj106 testdata.508089 2026-02-17T12:33:14.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj107 testdata.508089 2026-02-17T12:33:14.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj108 testdata.508089 2026-02-17T12:33:14.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj109 testdata.508089 2026-02-17T12:33:14.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj110 testdata.508089 2026-02-17T12:33:14.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj111 testdata.508089 2026-02-17T12:33:14.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj112 testdata.508089 2026-02-17T12:33:14.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj113 testdata.508089 2026-02-17T12:33:14.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj114 testdata.508089 2026-02-17T12:33:14.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj115 testdata.508089 2026-02-17T12:33:14.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj116 testdata.508089 2026-02-17T12:33:14.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj117 testdata.508089 2026-02-17T12:33:14.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj118 testdata.508089 2026-02-17T12:33:14.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj119 testdata.508089 2026-02-17T12:33:14.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj120 testdata.508089 2026-02-17T12:33:14.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj121 testdata.508089 2026-02-17T12:33:14.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj122 testdata.508089 2026-02-17T12:33:14.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj123 testdata.508089 2026-02-17T12:33:14.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj124 testdata.508089 2026-02-17T12:33:14.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj125 testdata.508089 2026-02-17T12:33:14.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj126 testdata.508089 2026-02-17T12:33:14.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj127 testdata.508089 2026-02-17T12:33:14.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj128 testdata.508089 2026-02-17T12:33:14.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj129 testdata.508089 2026-02-17T12:33:14.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj130 testdata.508089 2026-02-17T12:33:14.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj131 testdata.508089 2026-02-17T12:33:14.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj132 testdata.508089 2026-02-17T12:33:14.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj133 testdata.508089 2026-02-17T12:33:14.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj134 testdata.508089 2026-02-17T12:33:14.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj135 testdata.508089 2026-02-17T12:33:14.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj136 testdata.508089 2026-02-17T12:33:14.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj137 testdata.508089 2026-02-17T12:33:14.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj138 testdata.508089 2026-02-17T12:33:14.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj139 testdata.508089 2026-02-17T12:33:14.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj140 testdata.508089 2026-02-17T12:33:14.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj141 testdata.508089 2026-02-17T12:33:14.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:14.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj142 testdata.508089 2026-02-17T12:33:15.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj143 testdata.508089 2026-02-17T12:33:15.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj144 testdata.508089 2026-02-17T12:33:15.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj145 testdata.508089 2026-02-17T12:33:15.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj146 testdata.508089 2026-02-17T12:33:15.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj147 testdata.508089 2026-02-17T12:33:15.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj148 testdata.508089 2026-02-17T12:33:15.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj149 testdata.508089 2026-02-17T12:33:15.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj150 testdata.508089 2026-02-17T12:33:15.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj151 testdata.508089 2026-02-17T12:33:15.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj152 testdata.508089 2026-02-17T12:33:15.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj153 testdata.508089 2026-02-17T12:33:15.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj154 testdata.508089 2026-02-17T12:33:15.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj155 testdata.508089 2026-02-17T12:33:15.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj156 testdata.508089 2026-02-17T12:33:15.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj157 testdata.508089 2026-02-17T12:33:15.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj158 testdata.508089 2026-02-17T12:33:15.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj159 testdata.508089 2026-02-17T12:33:15.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj160 testdata.508089 2026-02-17T12:33:15.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj161 testdata.508089 2026-02-17T12:33:15.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj162 testdata.508089 2026-02-17T12:33:15.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj163 testdata.508089 2026-02-17T12:33:15.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj164 testdata.508089 2026-02-17T12:33:15.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj165 testdata.508089 2026-02-17T12:33:15.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj166 testdata.508089 2026-02-17T12:33:15.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj167 testdata.508089 2026-02-17T12:33:15.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj168 testdata.508089 2026-02-17T12:33:15.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj169 testdata.508089 2026-02-17T12:33:15.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj170 testdata.508089 2026-02-17T12:33:15.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj171 testdata.508089 2026-02-17T12:33:15.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj172 testdata.508089 2026-02-17T12:33:15.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj173 testdata.508089 2026-02-17T12:33:15.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj174 testdata.508089 2026-02-17T12:33:15.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj175 testdata.508089 2026-02-17T12:33:15.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj176 testdata.508089 2026-02-17T12:33:15.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj177 testdata.508089 2026-02-17T12:33:15.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj178 testdata.508089 2026-02-17T12:33:15.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj179 testdata.508089 2026-02-17T12:33:15.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj180 testdata.508089 2026-02-17T12:33:15.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj181 testdata.508089 2026-02-17T12:33:15.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj182 testdata.508089 2026-02-17T12:33:15.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj183 testdata.508089 2026-02-17T12:33:15.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj184 testdata.508089 2026-02-17T12:33:15.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:15.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj185 testdata.508089 2026-02-17T12:33:16.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj186 testdata.508089 2026-02-17T12:33:16.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj187 testdata.508089 2026-02-17T12:33:16.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj188 testdata.508089 2026-02-17T12:33:16.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj189 testdata.508089 2026-02-17T12:33:16.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj190 testdata.508089 2026-02-17T12:33:16.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj191 testdata.508089 2026-02-17T12:33:16.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj192 testdata.508089 2026-02-17T12:33:16.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj193 testdata.508089 2026-02-17T12:33:16.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj194 testdata.508089 2026-02-17T12:33:16.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj195 testdata.508089 2026-02-17T12:33:16.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj196 testdata.508089 2026-02-17T12:33:16.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj197 testdata.508089 2026-02-17T12:33:16.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj198 testdata.508089 2026-02-17T12:33:16.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj199 testdata.508089 2026-02-17T12:33:16.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj200 testdata.508089 2026-02-17T12:33:16.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj201 testdata.508089 2026-02-17T12:33:16.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj202 testdata.508089 2026-02-17T12:33:16.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj203 testdata.508089 2026-02-17T12:33:16.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj204 testdata.508089 2026-02-17T12:33:16.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj205 testdata.508089 2026-02-17T12:33:16.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj206 testdata.508089 2026-02-17T12:33:16.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj207 testdata.508089 2026-02-17T12:33:16.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj208 testdata.508089 2026-02-17T12:33:16.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj209 testdata.508089 2026-02-17T12:33:16.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj210 testdata.508089 2026-02-17T12:33:16.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj211 testdata.508089 2026-02-17T12:33:16.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj212 testdata.508089 2026-02-17T12:33:16.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj213 testdata.508089 2026-02-17T12:33:16.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj214 testdata.508089 2026-02-17T12:33:16.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj215 testdata.508089 2026-02-17T12:33:16.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj216 testdata.508089 2026-02-17T12:33:16.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj217 testdata.508089 2026-02-17T12:33:16.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj218 testdata.508089 2026-02-17T12:33:16.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj219 testdata.508089 2026-02-17T12:33:16.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj220 testdata.508089 2026-02-17T12:33:16.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj221 testdata.508089 2026-02-17T12:33:16.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj222 testdata.508089 2026-02-17T12:33:16.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj223 testdata.508089 2026-02-17T12:33:16.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj224 testdata.508089 2026-02-17T12:33:16.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj225 testdata.508089 2026-02-17T12:33:16.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj226 testdata.508089 2026-02-17T12:33:16.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:16.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj227 testdata.508089 2026-02-17T12:33:17.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj228 testdata.508089 2026-02-17T12:33:17.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj229 testdata.508089 2026-02-17T12:33:17.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj230 testdata.508089 2026-02-17T12:33:17.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj231 testdata.508089 2026-02-17T12:33:17.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj232 testdata.508089 2026-02-17T12:33:17.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj233 testdata.508089 2026-02-17T12:33:17.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj234 testdata.508089 2026-02-17T12:33:17.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj235 testdata.508089 2026-02-17T12:33:17.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj236 testdata.508089 2026-02-17T12:33:17.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj237 testdata.508089 2026-02-17T12:33:17.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj238 testdata.508089 2026-02-17T12:33:17.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj239 testdata.508089 2026-02-17T12:33:17.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj240 testdata.508089 2026-02-17T12:33:17.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj241 testdata.508089 2026-02-17T12:33:17.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj242 testdata.508089 2026-02-17T12:33:17.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj243 testdata.508089 2026-02-17T12:33:17.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj244 testdata.508089 2026-02-17T12:33:17.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj245 testdata.508089 2026-02-17T12:33:17.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj246 testdata.508089 2026-02-17T12:33:17.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj247 testdata.508089 2026-02-17T12:33:17.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj248 testdata.508089 2026-02-17T12:33:17.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj249 testdata.508089 2026-02-17T12:33:17.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj250 testdata.508089 2026-02-17T12:33:17.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj251 testdata.508089 2026-02-17T12:33:17.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj252 testdata.508089 2026-02-17T12:33:17.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj253 testdata.508089 2026-02-17T12:33:17.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj254 testdata.508089 2026-02-17T12:33:17.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj255 testdata.508089 2026-02-17T12:33:17.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj256 testdata.508089 2026-02-17T12:33:17.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj257 testdata.508089 2026-02-17T12:33:17.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj258 testdata.508089 2026-02-17T12:33:17.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj259 testdata.508089 2026-02-17T12:33:17.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj260 testdata.508089 2026-02-17T12:33:17.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj261 testdata.508089 2026-02-17T12:33:17.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj262 testdata.508089 2026-02-17T12:33:17.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj263 testdata.508089 2026-02-17T12:33:17.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj264 testdata.508089 2026-02-17T12:33:17.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj265 testdata.508089 2026-02-17T12:33:17.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj266 testdata.508089 2026-02-17T12:33:17.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj267 testdata.508089 2026-02-17T12:33:17.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj268 testdata.508089 2026-02-17T12:33:17.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:17.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj269 testdata.508089 2026-02-17T12:33:18.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj270 testdata.508089 2026-02-17T12:33:18.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj271 testdata.508089 2026-02-17T12:33:18.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj272 testdata.508089 2026-02-17T12:33:18.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj273 testdata.508089 2026-02-17T12:33:18.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj274 testdata.508089 2026-02-17T12:33:18.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj275 testdata.508089 2026-02-17T12:33:18.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj276 testdata.508089 2026-02-17T12:33:18.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj277 testdata.508089 2026-02-17T12:33:18.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj278 testdata.508089 2026-02-17T12:33:18.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.223 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj279 testdata.508089 2026-02-17T12:33:18.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj280 testdata.508089 2026-02-17T12:33:18.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj281 testdata.508089 2026-02-17T12:33:18.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj282 testdata.508089 2026-02-17T12:33:18.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj283 testdata.508089 2026-02-17T12:33:18.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj284 testdata.508089 2026-02-17T12:33:18.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj285 testdata.508089 2026-02-17T12:33:18.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj286 testdata.508089 2026-02-17T12:33:18.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj287 testdata.508089 2026-02-17T12:33:18.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj288 testdata.508089 2026-02-17T12:33:18.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj289 testdata.508089 2026-02-17T12:33:18.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj290 testdata.508089 2026-02-17T12:33:18.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj291 testdata.508089 2026-02-17T12:33:18.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj292 testdata.508089 2026-02-17T12:33:18.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj293 testdata.508089 2026-02-17T12:33:18.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj294 testdata.508089 2026-02-17T12:33:18.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj295 testdata.508089 2026-02-17T12:33:18.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj296 testdata.508089 2026-02-17T12:33:18.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj297 testdata.508089 2026-02-17T12:33:18.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj298 testdata.508089 2026-02-17T12:33:18.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj299 testdata.508089 2026-02-17T12:33:18.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj300 testdata.508089 2026-02-17T12:33:18.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj301 testdata.508089 2026-02-17T12:33:18.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj302 testdata.508089 2026-02-17T12:33:18.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj303 testdata.508089 2026-02-17T12:33:18.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj304 testdata.508089 2026-02-17T12:33:18.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj305 testdata.508089 2026-02-17T12:33:18.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj306 testdata.508089 2026-02-17T12:33:18.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj307 testdata.508089 2026-02-17T12:33:18.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj308 testdata.508089 2026-02-17T12:33:18.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj309 testdata.508089 2026-02-17T12:33:18.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj310 testdata.508089 2026-02-17T12:33:18.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj311 testdata.508089 2026-02-17T12:33:18.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:18.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj312 testdata.508089 2026-02-17T12:33:19.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.013 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj313 testdata.508089 2026-02-17T12:33:19.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj314 testdata.508089 2026-02-17T12:33:19.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj315 testdata.508089 2026-02-17T12:33:19.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj316 testdata.508089 2026-02-17T12:33:19.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj317 testdata.508089 2026-02-17T12:33:19.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.127 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj318 testdata.508089 2026-02-17T12:33:19.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj319 testdata.508089 2026-02-17T12:33:19.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj320 testdata.508089 2026-02-17T12:33:19.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj321 testdata.508089 2026-02-17T12:33:19.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj322 testdata.508089 2026-02-17T12:33:19.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj323 testdata.508089 2026-02-17T12:33:19.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj324 testdata.508089 2026-02-17T12:33:19.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj325 testdata.508089 2026-02-17T12:33:19.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj326 testdata.508089 2026-02-17T12:33:19.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj327 testdata.508089 2026-02-17T12:33:19.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj328 testdata.508089 2026-02-17T12:33:19.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj329 testdata.508089 2026-02-17T12:33:19.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj330 testdata.508089 2026-02-17T12:33:19.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj331 testdata.508089 2026-02-17T12:33:19.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj332 testdata.508089 2026-02-17T12:33:19.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj333 testdata.508089 2026-02-17T12:33:19.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj334 testdata.508089 2026-02-17T12:33:19.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj335 testdata.508089 2026-02-17T12:33:19.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj336 testdata.508089 2026-02-17T12:33:19.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj337 testdata.508089 2026-02-17T12:33:19.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj338 testdata.508089 2026-02-17T12:33:19.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj339 testdata.508089 2026-02-17T12:33:19.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj340 testdata.508089 2026-02-17T12:33:19.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj341 testdata.508089 2026-02-17T12:33:19.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.690 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj342 testdata.508089 2026-02-17T12:33:19.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj343 testdata.508089 2026-02-17T12:33:19.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj344 testdata.508089 2026-02-17T12:33:19.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj345 testdata.508089 2026-02-17T12:33:19.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj346 testdata.508089 2026-02-17T12:33:19.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj347 testdata.508089 2026-02-17T12:33:19.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj348 testdata.508089 2026-02-17T12:33:19.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj349 testdata.508089 2026-02-17T12:33:19.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj350 testdata.508089 2026-02-17T12:33:19.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj351 testdata.508089 2026-02-17T12:33:19.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj352 testdata.508089 2026-02-17T12:33:19.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj353 testdata.508089 2026-02-17T12:33:19.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj354 testdata.508089 2026-02-17T12:33:19.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:19.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj355 testdata.508089 2026-02-17T12:33:20.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj356 testdata.508089 2026-02-17T12:33:20.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj357 testdata.508089 2026-02-17T12:33:20.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj358 testdata.508089 2026-02-17T12:33:20.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj359 testdata.508089 2026-02-17T12:33:20.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj360 testdata.508089 2026-02-17T12:33:20.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj361 testdata.508089 2026-02-17T12:33:20.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj362 testdata.508089 2026-02-17T12:33:20.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj363 testdata.508089 2026-02-17T12:33:20.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj364 testdata.508089 2026-02-17T12:33:20.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj365 testdata.508089 2026-02-17T12:33:20.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj366 testdata.508089 2026-02-17T12:33:20.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj367 testdata.508089 2026-02-17T12:33:20.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj368 testdata.508089 2026-02-17T12:33:20.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj369 testdata.508089 2026-02-17T12:33:20.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj370 testdata.508089 2026-02-17T12:33:20.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj371 testdata.508089 2026-02-17T12:33:20.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj372 testdata.508089 2026-02-17T12:33:20.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj373 testdata.508089 2026-02-17T12:33:20.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj374 testdata.508089 2026-02-17T12:33:20.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj375 testdata.508089 2026-02-17T12:33:20.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj376 testdata.508089 2026-02-17T12:33:20.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj377 testdata.508089 2026-02-17T12:33:20.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj378 testdata.508089 2026-02-17T12:33:20.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj379 testdata.508089 2026-02-17T12:33:20.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj380 testdata.508089 2026-02-17T12:33:20.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj381 testdata.508089 2026-02-17T12:33:20.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj382 testdata.508089 2026-02-17T12:33:20.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj383 testdata.508089 2026-02-17T12:33:20.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj384 testdata.508089 2026-02-17T12:33:20.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj385 testdata.508089 2026-02-17T12:33:20.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj386 testdata.508089 2026-02-17T12:33:20.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj387 testdata.508089 2026-02-17T12:33:20.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj388 testdata.508089 2026-02-17T12:33:20.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj389 testdata.508089 2026-02-17T12:33:20.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj390 testdata.508089 2026-02-17T12:33:20.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj391 testdata.508089 2026-02-17T12:33:20.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj392 testdata.508089 2026-02-17T12:33:20.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj393 testdata.508089 2026-02-17T12:33:20.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj394 testdata.508089 2026-02-17T12:33:20.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.931 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj395 testdata.508089 2026-02-17T12:33:20.955 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj396 testdata.508089 2026-02-17T12:33:20.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:20.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj397 testdata.508089 2026-02-17T12:33:21.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj398 testdata.508089 2026-02-17T12:33:21.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj399 testdata.508089 2026-02-17T12:33:21.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj400 testdata.508089 2026-02-17T12:33:21.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj401 testdata.508089 2026-02-17T12:33:21.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj402 testdata.508089 2026-02-17T12:33:21.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj403 testdata.508089 2026-02-17T12:33:21.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj404 testdata.508089 2026-02-17T12:33:21.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj405 testdata.508089 2026-02-17T12:33:21.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.197 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj406 testdata.508089 2026-02-17T12:33:21.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj407 testdata.508089 2026-02-17T12:33:21.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj408 testdata.508089 2026-02-17T12:33:21.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.270 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj409 testdata.508089 2026-02-17T12:33:21.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj410 testdata.508089 2026-02-17T12:33:21.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj411 testdata.508089 2026-02-17T12:33:21.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj412 testdata.508089 2026-02-17T12:33:21.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj413 testdata.508089 2026-02-17T12:33:21.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj414 testdata.508089 2026-02-17T12:33:21.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj415 testdata.508089 2026-02-17T12:33:21.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj416 testdata.508089 2026-02-17T12:33:21.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj417 testdata.508089 2026-02-17T12:33:21.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj418 testdata.508089 2026-02-17T12:33:21.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj419 testdata.508089 2026-02-17T12:33:21.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj420 testdata.508089 2026-02-17T12:33:21.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj421 testdata.508089 2026-02-17T12:33:21.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj422 testdata.508089 2026-02-17T12:33:21.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj423 testdata.508089 2026-02-17T12:33:21.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj424 testdata.508089 2026-02-17T12:33:21.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj425 testdata.508089 2026-02-17T12:33:21.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj426 testdata.508089 2026-02-17T12:33:21.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj427 testdata.508089 2026-02-17T12:33:21.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj428 testdata.508089 2026-02-17T12:33:21.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj429 testdata.508089 2026-02-17T12:33:21.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj430 testdata.508089 2026-02-17T12:33:21.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj431 testdata.508089 2026-02-17T12:33:21.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj432 testdata.508089 2026-02-17T12:33:21.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.837 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj433 testdata.508089 2026-02-17T12:33:21.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj434 testdata.508089 2026-02-17T12:33:21.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj435 testdata.508089 2026-02-17T12:33:21.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj436 testdata.508089 2026-02-17T12:33:21.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj437 testdata.508089 2026-02-17T12:33:21.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj438 testdata.508089 2026-02-17T12:33:21.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj439 testdata.508089 2026-02-17T12:33:21.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:21.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj440 testdata.508089 2026-02-17T12:33:22.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj441 testdata.508089 2026-02-17T12:33:22.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj442 testdata.508089 2026-02-17T12:33:22.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj443 testdata.508089 2026-02-17T12:33:22.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj444 testdata.508089 2026-02-17T12:33:22.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj445 testdata.508089 2026-02-17T12:33:22.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj446 testdata.508089 2026-02-17T12:33:22.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj447 testdata.508089 2026-02-17T12:33:22.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj448 testdata.508089 2026-02-17T12:33:22.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj449 testdata.508089 2026-02-17T12:33:22.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj450 testdata.508089 2026-02-17T12:33:22.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj451 testdata.508089 2026-02-17T12:33:22.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj452 testdata.508089 2026-02-17T12:33:22.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj453 testdata.508089 2026-02-17T12:33:22.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj454 testdata.508089 2026-02-17T12:33:22.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj455 testdata.508089 2026-02-17T12:33:22.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj456 testdata.508089 2026-02-17T12:33:22.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj457 testdata.508089 2026-02-17T12:33:22.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj458 testdata.508089 2026-02-17T12:33:22.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj459 testdata.508089 2026-02-17T12:33:22.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj460 testdata.508089 2026-02-17T12:33:22.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj461 testdata.508089 2026-02-17T12:33:22.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj462 testdata.508089 2026-02-17T12:33:22.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj463 testdata.508089 2026-02-17T12:33:22.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj464 testdata.508089 2026-02-17T12:33:22.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj465 testdata.508089 2026-02-17T12:33:22.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj466 testdata.508089 2026-02-17T12:33:22.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj467 testdata.508089 2026-02-17T12:33:22.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj468 testdata.508089 2026-02-17T12:33:22.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj469 testdata.508089 2026-02-17T12:33:22.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj470 testdata.508089 2026-02-17T12:33:22.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj471 testdata.508089 2026-02-17T12:33:22.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj472 testdata.508089 2026-02-17T12:33:22.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj473 testdata.508089 2026-02-17T12:33:22.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj474 testdata.508089 2026-02-17T12:33:22.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj475 testdata.508089 2026-02-17T12:33:22.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj476 testdata.508089 2026-02-17T12:33:22.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj477 testdata.508089 2026-02-17T12:33:22.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj478 testdata.508089 2026-02-17T12:33:22.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj479 testdata.508089 2026-02-17T12:33:22.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj480 testdata.508089 2026-02-17T12:33:22.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj481 testdata.508089 2026-02-17T12:33:22.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:22.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj482 testdata.508089 2026-02-17T12:33:23.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj483 testdata.508089 2026-02-17T12:33:23.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj484 testdata.508089 2026-02-17T12:33:23.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj485 testdata.508089 2026-02-17T12:33:23.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj486 testdata.508089 2026-02-17T12:33:23.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj487 testdata.508089 2026-02-17T12:33:23.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.130 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj488 testdata.508089 2026-02-17T12:33:23.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj489 testdata.508089 2026-02-17T12:33:23.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj490 testdata.508089 2026-02-17T12:33:23.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj491 testdata.508089 2026-02-17T12:33:23.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj492 testdata.508089 2026-02-17T12:33:23.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj493 testdata.508089 2026-02-17T12:33:23.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj494 testdata.508089 2026-02-17T12:33:23.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj495 testdata.508089 2026-02-17T12:33:23.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj496 testdata.508089 2026-02-17T12:33:23.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj497 testdata.508089 2026-02-17T12:33:23.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.364 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj498 testdata.508089 2026-02-17T12:33:23.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj499 testdata.508089 2026-02-17T12:33:23.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj500 testdata.508089 2026-02-17T12:33:23.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj501 testdata.508089 2026-02-17T12:33:23.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj502 testdata.508089 2026-02-17T12:33:23.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj503 testdata.508089 2026-02-17T12:33:23.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.510 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj504 testdata.508089 2026-02-17T12:33:23.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj505 testdata.508089 2026-02-17T12:33:23.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj506 testdata.508089 2026-02-17T12:33:23.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj507 testdata.508089 2026-02-17T12:33:23.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj508 testdata.508089 2026-02-17T12:33:23.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj509 testdata.508089 2026-02-17T12:33:23.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj510 testdata.508089 2026-02-17T12:33:23.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj511 testdata.508089 2026-02-17T12:33:23.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj512 testdata.508089 2026-02-17T12:33:23.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj513 testdata.508089 2026-02-17T12:33:23.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj514 testdata.508089 2026-02-17T12:33:23.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj515 testdata.508089 2026-02-17T12:33:23.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj516 testdata.508089 2026-02-17T12:33:23.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj517 testdata.508089 2026-02-17T12:33:23.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj518 testdata.508089 2026-02-17T12:33:23.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj519 testdata.508089 2026-02-17T12:33:23.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj520 testdata.508089 2026-02-17T12:33:23.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj521 testdata.508089 2026-02-17T12:33:23.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj522 testdata.508089 2026-02-17T12:33:23.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj523 testdata.508089 2026-02-17T12:33:23.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:23.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj524 testdata.508089 2026-02-17T12:33:24.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj525 testdata.508089 2026-02-17T12:33:24.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj526 testdata.508089 2026-02-17T12:33:24.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj527 testdata.508089 2026-02-17T12:33:24.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj528 testdata.508089 2026-02-17T12:33:24.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj529 testdata.508089 2026-02-17T12:33:24.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj530 testdata.508089 2026-02-17T12:33:24.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj531 testdata.508089 2026-02-17T12:33:24.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj532 testdata.508089 2026-02-17T12:33:24.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj533 testdata.508089 2026-02-17T12:33:24.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj534 testdata.508089 2026-02-17T12:33:24.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj535 testdata.508089 2026-02-17T12:33:24.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj536 testdata.508089 2026-02-17T12:33:24.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj537 testdata.508089 2026-02-17T12:33:24.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj538 testdata.508089 2026-02-17T12:33:24.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj539 testdata.508089 2026-02-17T12:33:24.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj540 testdata.508089 2026-02-17T12:33:24.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj541 testdata.508089 2026-02-17T12:33:24.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj542 testdata.508089 2026-02-17T12:33:24.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj543 testdata.508089 2026-02-17T12:33:24.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj544 testdata.508089 2026-02-17T12:33:24.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj545 testdata.508089 2026-02-17T12:33:24.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj546 testdata.508089 2026-02-17T12:33:24.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj547 testdata.508089 2026-02-17T12:33:24.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj548 testdata.508089 2026-02-17T12:33:24.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj549 testdata.508089 2026-02-17T12:33:24.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj550 testdata.508089 2026-02-17T12:33:24.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj551 testdata.508089 2026-02-17T12:33:24.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj552 testdata.508089 2026-02-17T12:33:24.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj553 testdata.508089 2026-02-17T12:33:24.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.599 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj554 testdata.508089 2026-02-17T12:33:24.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj555 testdata.508089 2026-02-17T12:33:24.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj556 testdata.508089 2026-02-17T12:33:24.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj557 testdata.508089 2026-02-17T12:33:24.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj558 testdata.508089 2026-02-17T12:33:24.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj559 testdata.508089 2026-02-17T12:33:24.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj560 testdata.508089 2026-02-17T12:33:24.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj561 testdata.508089 2026-02-17T12:33:24.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj562 testdata.508089 2026-02-17T12:33:24.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj563 testdata.508089 2026-02-17T12:33:24.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj564 testdata.508089 2026-02-17T12:33:24.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj565 testdata.508089 2026-02-17T12:33:24.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj566 testdata.508089 2026-02-17T12:33:24.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj567 testdata.508089 2026-02-17T12:33:24.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj568 testdata.508089 2026-02-17T12:33:24.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj569 testdata.508089 2026-02-17T12:33:24.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj570 testdata.508089 2026-02-17T12:33:24.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj571 testdata.508089 2026-02-17T12:33:24.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:24.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj572 testdata.508089 2026-02-17T12:33:25.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj573 testdata.508089 2026-02-17T12:33:25.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj574 testdata.508089 2026-02-17T12:33:25.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj575 testdata.508089 2026-02-17T12:33:25.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj576 testdata.508089 2026-02-17T12:33:25.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj577 testdata.508089 2026-02-17T12:33:25.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj578 testdata.508089 2026-02-17T12:33:25.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj579 testdata.508089 2026-02-17T12:33:25.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj580 testdata.508089 2026-02-17T12:33:25.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj581 testdata.508089 2026-02-17T12:33:25.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj582 testdata.508089 2026-02-17T12:33:25.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj583 testdata.508089 2026-02-17T12:33:25.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj584 testdata.508089 2026-02-17T12:33:25.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj585 testdata.508089 2026-02-17T12:33:25.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj586 testdata.508089 2026-02-17T12:33:25.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj587 testdata.508089 2026-02-17T12:33:25.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.374 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj588 testdata.508089 2026-02-17T12:33:25.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj589 testdata.508089 2026-02-17T12:33:25.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj590 testdata.508089 2026-02-17T12:33:25.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj591 testdata.508089 2026-02-17T12:33:25.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj592 testdata.508089 2026-02-17T12:33:25.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj593 testdata.508089 2026-02-17T12:33:25.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj594 testdata.508089 2026-02-17T12:33:25.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj595 testdata.508089 2026-02-17T12:33:25.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj596 testdata.508089 2026-02-17T12:33:25.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj597 testdata.508089 2026-02-17T12:33:25.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj598 testdata.508089 2026-02-17T12:33:25.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj599 testdata.508089 2026-02-17T12:33:25.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj600 testdata.508089 2026-02-17T12:33:25.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj601 testdata.508089 2026-02-17T12:33:25.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj602 testdata.508089 2026-02-17T12:33:25.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj603 testdata.508089 2026-02-17T12:33:25.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj604 testdata.508089 2026-02-17T12:33:25.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj605 testdata.508089 2026-02-17T12:33:25.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj606 testdata.508089 2026-02-17T12:33:25.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj607 testdata.508089 2026-02-17T12:33:25.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj608 testdata.508089 2026-02-17T12:33:25.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj609 testdata.508089 2026-02-17T12:33:25.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj610 testdata.508089 2026-02-17T12:33:25.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj611 testdata.508089 2026-02-17T12:33:25.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj612 testdata.508089 2026-02-17T12:33:25.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj613 testdata.508089 2026-02-17T12:33:25.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj614 testdata.508089 2026-02-17T12:33:25.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj615 testdata.508089 2026-02-17T12:33:25.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:25.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj616 testdata.508089 2026-02-17T12:33:26.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj617 testdata.508089 2026-02-17T12:33:26.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj618 testdata.508089 2026-02-17T12:33:26.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj619 testdata.508089 2026-02-17T12:33:26.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj620 testdata.508089 2026-02-17T12:33:26.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj621 testdata.508089 2026-02-17T12:33:26.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj622 testdata.508089 2026-02-17T12:33:26.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj623 testdata.508089 2026-02-17T12:33:26.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj624 testdata.508089 2026-02-17T12:33:26.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj625 testdata.508089 2026-02-17T12:33:26.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj626 testdata.508089 2026-02-17T12:33:26.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj627 testdata.508089 2026-02-17T12:33:26.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj628 testdata.508089 2026-02-17T12:33:26.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj629 testdata.508089 2026-02-17T12:33:26.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj630 testdata.508089 2026-02-17T12:33:26.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj631 testdata.508089 2026-02-17T12:33:26.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj632 testdata.508089 2026-02-17T12:33:26.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj633 testdata.508089 2026-02-17T12:33:26.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj634 testdata.508089 2026-02-17T12:33:26.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj635 testdata.508089 2026-02-17T12:33:26.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj636 testdata.508089 2026-02-17T12:33:26.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj637 testdata.508089 2026-02-17T12:33:26.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj638 testdata.508089 2026-02-17T12:33:26.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj639 testdata.508089 2026-02-17T12:33:26.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj640 testdata.508089 2026-02-17T12:33:26.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj641 testdata.508089 2026-02-17T12:33:26.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj642 testdata.508089 2026-02-17T12:33:26.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj643 testdata.508089 2026-02-17T12:33:26.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj644 testdata.508089 2026-02-17T12:33:26.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj645 testdata.508089 2026-02-17T12:33:26.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj646 testdata.508089 2026-02-17T12:33:26.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj647 testdata.508089 2026-02-17T12:33:26.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj648 testdata.508089 2026-02-17T12:33:26.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj649 testdata.508089 2026-02-17T12:33:26.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj650 testdata.508089 2026-02-17T12:33:26.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj651 testdata.508089 2026-02-17T12:33:26.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj652 testdata.508089 2026-02-17T12:33:26.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj653 testdata.508089 2026-02-17T12:33:26.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.906 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj654 testdata.508089 2026-02-17T12:33:26.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj655 testdata.508089 2026-02-17T12:33:26.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj656 testdata.508089 2026-02-17T12:33:26.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:26.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj657 testdata.508089 2026-02-17T12:33:27.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj658 testdata.508089 2026-02-17T12:33:27.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj659 testdata.508089 2026-02-17T12:33:27.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj660 testdata.508089 2026-02-17T12:33:27.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj661 testdata.508089 2026-02-17T12:33:27.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj662 testdata.508089 2026-02-17T12:33:27.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj663 testdata.508089 2026-02-17T12:33:27.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj664 testdata.508089 2026-02-17T12:33:27.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj665 testdata.508089 2026-02-17T12:33:27.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj666 testdata.508089 2026-02-17T12:33:27.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.217 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj667 testdata.508089 2026-02-17T12:33:27.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj668 testdata.508089 2026-02-17T12:33:27.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj669 testdata.508089 2026-02-17T12:33:27.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj670 testdata.508089 2026-02-17T12:33:27.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj671 testdata.508089 2026-02-17T12:33:27.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj672 testdata.508089 2026-02-17T12:33:27.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj673 testdata.508089 2026-02-17T12:33:27.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj674 testdata.508089 2026-02-17T12:33:27.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj675 testdata.508089 2026-02-17T12:33:27.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.429 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj676 testdata.508089 2026-02-17T12:33:27.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj677 testdata.508089 2026-02-17T12:33:27.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj678 testdata.508089 2026-02-17T12:33:27.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj679 testdata.508089 2026-02-17T12:33:27.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj680 testdata.508089 2026-02-17T12:33:27.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj681 testdata.508089 2026-02-17T12:33:27.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj682 testdata.508089 2026-02-17T12:33:27.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj683 testdata.508089 2026-02-17T12:33:27.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj684 testdata.508089 2026-02-17T12:33:27.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj685 testdata.508089 2026-02-17T12:33:27.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj686 testdata.508089 2026-02-17T12:33:27.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj687 testdata.508089 2026-02-17T12:33:27.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj688 testdata.508089 2026-02-17T12:33:27.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj689 testdata.508089 2026-02-17T12:33:27.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj690 testdata.508089 2026-02-17T12:33:27.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj691 testdata.508089 2026-02-17T12:33:27.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj692 testdata.508089 2026-02-17T12:33:27.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj693 testdata.508089 2026-02-17T12:33:27.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj694 testdata.508089 2026-02-17T12:33:27.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj695 testdata.508089 2026-02-17T12:33:27.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj696 testdata.508089 2026-02-17T12:33:27.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj697 testdata.508089 2026-02-17T12:33:27.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj698 testdata.508089 2026-02-17T12:33:27.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:27.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj699 testdata.508089 2026-02-17T12:33:28.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj700 testdata.508089 2026-02-17T12:33:28.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj701 testdata.508089 2026-02-17T12:33:28.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj702 testdata.508089 2026-02-17T12:33:28.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj703 testdata.508089 2026-02-17T12:33:28.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj704 testdata.508089 2026-02-17T12:33:28.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj705 testdata.508089 2026-02-17T12:33:28.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj706 testdata.508089 2026-02-17T12:33:28.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj707 testdata.508089 2026-02-17T12:33:28.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj708 testdata.508089 2026-02-17T12:33:28.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj709 testdata.508089 2026-02-17T12:33:28.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj710 testdata.508089 2026-02-17T12:33:28.266 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj711 testdata.508089 2026-02-17T12:33:28.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj712 testdata.508089 2026-02-17T12:33:28.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj713 testdata.508089 2026-02-17T12:33:28.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj714 testdata.508089 2026-02-17T12:33:28.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj715 testdata.508089 2026-02-17T12:33:28.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj716 testdata.508089 2026-02-17T12:33:28.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj717 testdata.508089 2026-02-17T12:33:28.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj718 testdata.508089 2026-02-17T12:33:28.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.444 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj719 testdata.508089 2026-02-17T12:33:28.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj720 testdata.508089 2026-02-17T12:33:28.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj721 testdata.508089 2026-02-17T12:33:28.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj722 testdata.508089 2026-02-17T12:33:28.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj723 testdata.508089 2026-02-17T12:33:28.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj724 testdata.508089 2026-02-17T12:33:28.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj725 testdata.508089 2026-02-17T12:33:28.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj726 testdata.508089 2026-02-17T12:33:28.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj727 testdata.508089 2026-02-17T12:33:28.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj728 testdata.508089 2026-02-17T12:33:28.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj729 testdata.508089 2026-02-17T12:33:28.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj730 testdata.508089 2026-02-17T12:33:28.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj731 testdata.508089 2026-02-17T12:33:28.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj732 testdata.508089 2026-02-17T12:33:28.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj733 testdata.508089 2026-02-17T12:33:28.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj734 testdata.508089 2026-02-17T12:33:28.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj735 testdata.508089 2026-02-17T12:33:28.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj736 testdata.508089 2026-02-17T12:33:28.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj737 testdata.508089 2026-02-17T12:33:28.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj738 testdata.508089 2026-02-17T12:33:28.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj739 testdata.508089 2026-02-17T12:33:28.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj740 testdata.508089 2026-02-17T12:33:28.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj741 testdata.508089 2026-02-17T12:33:28.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj742 testdata.508089 2026-02-17T12:33:28.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:28.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj743 testdata.508089 2026-02-17T12:33:29.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj744 testdata.508089 2026-02-17T12:33:29.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj745 testdata.508089 2026-02-17T12:33:29.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj746 testdata.508089 2026-02-17T12:33:29.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj747 testdata.508089 2026-02-17T12:33:29.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj748 testdata.508089 2026-02-17T12:33:29.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj749 testdata.508089 2026-02-17T12:33:29.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj750 testdata.508089 2026-02-17T12:33:29.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj751 testdata.508089 2026-02-17T12:33:29.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj752 testdata.508089 2026-02-17T12:33:29.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj753 testdata.508089 2026-02-17T12:33:29.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj754 testdata.508089 2026-02-17T12:33:29.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj755 testdata.508089 2026-02-17T12:33:29.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj756 testdata.508089 2026-02-17T12:33:29.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj757 testdata.508089 2026-02-17T12:33:29.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj758 testdata.508089 2026-02-17T12:33:29.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj759 testdata.508089 2026-02-17T12:33:29.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj760 testdata.508089 2026-02-17T12:33:29.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj761 testdata.508089 2026-02-17T12:33:29.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj762 testdata.508089 2026-02-17T12:33:29.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj763 testdata.508089 2026-02-17T12:33:29.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj764 testdata.508089 2026-02-17T12:33:29.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj765 testdata.508089 2026-02-17T12:33:29.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj766 testdata.508089 2026-02-17T12:33:29.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj767 testdata.508089 2026-02-17T12:33:29.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj768 testdata.508089 2026-02-17T12:33:29.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj769 testdata.508089 2026-02-17T12:33:29.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj770 testdata.508089 2026-02-17T12:33:29.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj771 testdata.508089 2026-02-17T12:33:29.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj772 testdata.508089 2026-02-17T12:33:29.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj773 testdata.508089 2026-02-17T12:33:29.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj774 testdata.508089 2026-02-17T12:33:29.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj775 testdata.508089 2026-02-17T12:33:29.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj776 testdata.508089 2026-02-17T12:33:29.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj777 testdata.508089 2026-02-17T12:33:29.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.799 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj778 testdata.508089 2026-02-17T12:33:29.823 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj779 testdata.508089 2026-02-17T12:33:29.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.848 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj780 testdata.508089 2026-02-17T12:33:29.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj781 testdata.508089 2026-02-17T12:33:29.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj782 testdata.508089 2026-02-17T12:33:29.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj783 testdata.508089 2026-02-17T12:33:29.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj784 testdata.508089 2026-02-17T12:33:29.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj785 testdata.508089 2026-02-17T12:33:29.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:29.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj786 testdata.508089 2026-02-17T12:33:30.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj787 testdata.508089 2026-02-17T12:33:30.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj788 testdata.508089 2026-02-17T12:33:30.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj789 testdata.508089 2026-02-17T12:33:30.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj790 testdata.508089 2026-02-17T12:33:30.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj791 testdata.508089 2026-02-17T12:33:30.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj792 testdata.508089 2026-02-17T12:33:30.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj793 testdata.508089 2026-02-17T12:33:30.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj794 testdata.508089 2026-02-17T12:33:30.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj795 testdata.508089 2026-02-17T12:33:30.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj796 testdata.508089 2026-02-17T12:33:30.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj797 testdata.508089 2026-02-17T12:33:30.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj798 testdata.508089 2026-02-17T12:33:30.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj799 testdata.508089 2026-02-17T12:33:30.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj800 testdata.508089 2026-02-17T12:33:30.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj801 testdata.508089 2026-02-17T12:33:30.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.375 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj802 testdata.508089 2026-02-17T12:33:30.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj803 testdata.508089 2026-02-17T12:33:30.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj804 testdata.508089 2026-02-17T12:33:30.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj805 testdata.508089 2026-02-17T12:33:30.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj806 testdata.508089 2026-02-17T12:33:30.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj807 testdata.508089 2026-02-17T12:33:30.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj808 testdata.508089 2026-02-17T12:33:30.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj809 testdata.508089 2026-02-17T12:33:30.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj810 testdata.508089 2026-02-17T12:33:30.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj811 testdata.508089 2026-02-17T12:33:30.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj812 testdata.508089 2026-02-17T12:33:30.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj813 testdata.508089 2026-02-17T12:33:30.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj814 testdata.508089 2026-02-17T12:33:30.688 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj815 testdata.508089 2026-02-17T12:33:30.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj816 testdata.508089 2026-02-17T12:33:30.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.737 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj817 testdata.508089 2026-02-17T12:33:30.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj818 testdata.508089 2026-02-17T12:33:30.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj819 testdata.508089 2026-02-17T12:33:30.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.808 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj820 testdata.508089 2026-02-17T12:33:30.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj821 testdata.508089 2026-02-17T12:33:30.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj822 testdata.508089 2026-02-17T12:33:30.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj823 testdata.508089 2026-02-17T12:33:30.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj824 testdata.508089 2026-02-17T12:33:30.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj825 testdata.508089 2026-02-17T12:33:30.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj826 testdata.508089 2026-02-17T12:33:30.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:30.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj827 testdata.508089 2026-02-17T12:33:31.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj828 testdata.508089 2026-02-17T12:33:31.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj829 testdata.508089 2026-02-17T12:33:31.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj830 testdata.508089 2026-02-17T12:33:31.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj831 testdata.508089 2026-02-17T12:33:31.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj832 testdata.508089 2026-02-17T12:33:31.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj833 testdata.508089 2026-02-17T12:33:31.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj834 testdata.508089 2026-02-17T12:33:31.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj835 testdata.508089 2026-02-17T12:33:31.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj836 testdata.508089 2026-02-17T12:33:31.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj837 testdata.508089 2026-02-17T12:33:31.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj838 testdata.508089 2026-02-17T12:33:31.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj839 testdata.508089 2026-02-17T12:33:31.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.283 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj840 testdata.508089 2026-02-17T12:33:31.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj841 testdata.508089 2026-02-17T12:33:31.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj842 testdata.508089 2026-02-17T12:33:31.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj843 testdata.508089 2026-02-17T12:33:31.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj844 testdata.508089 2026-02-17T12:33:31.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj845 testdata.508089 2026-02-17T12:33:31.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj846 testdata.508089 2026-02-17T12:33:31.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.430 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj847 testdata.508089 2026-02-17T12:33:31.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj848 testdata.508089 2026-02-17T12:33:31.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj849 testdata.508089 2026-02-17T12:33:31.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj850 testdata.508089 2026-02-17T12:33:31.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj851 testdata.508089 2026-02-17T12:33:31.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj852 testdata.508089 2026-02-17T12:33:31.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj853 testdata.508089 2026-02-17T12:33:31.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj854 testdata.508089 2026-02-17T12:33:31.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj855 testdata.508089 2026-02-17T12:33:31.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj856 testdata.508089 2026-02-17T12:33:31.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj857 testdata.508089 2026-02-17T12:33:31.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.674 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj858 testdata.508089 2026-02-17T12:33:31.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj859 testdata.508089 2026-02-17T12:33:31.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj860 testdata.508089 2026-02-17T12:33:31.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.739 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj861 testdata.508089 2026-02-17T12:33:31.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.758 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj862 testdata.508089 2026-02-17T12:33:31.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.775 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj863 testdata.508089 2026-02-17T12:33:31.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj864 testdata.508089 2026-02-17T12:33:31.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj865 testdata.508089 2026-02-17T12:33:31.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj866 testdata.508089 2026-02-17T12:33:31.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj867 testdata.508089 2026-02-17T12:33:31.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.892 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj868 testdata.508089 2026-02-17T12:33:31.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj869 testdata.508089 2026-02-17T12:33:31.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj870 testdata.508089 2026-02-17T12:33:31.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.963 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj871 testdata.508089 2026-02-17T12:33:31.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:31.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj872 testdata.508089 2026-02-17T12:33:32.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj873 testdata.508089 2026-02-17T12:33:32.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj874 testdata.508089 2026-02-17T12:33:32.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj875 testdata.508089 2026-02-17T12:33:32.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj876 testdata.508089 2026-02-17T12:33:32.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj877 testdata.508089 2026-02-17T12:33:32.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.133 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj878 testdata.508089 2026-02-17T12:33:32.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj879 testdata.508089 2026-02-17T12:33:32.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj880 testdata.508089 2026-02-17T12:33:32.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj881 testdata.508089 2026-02-17T12:33:32.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj882 testdata.508089 2026-02-17T12:33:32.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj883 testdata.508089 2026-02-17T12:33:32.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj884 testdata.508089 2026-02-17T12:33:32.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj885 testdata.508089 2026-02-17T12:33:32.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj886 testdata.508089 2026-02-17T12:33:32.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj887 testdata.508089 2026-02-17T12:33:32.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.366 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj888 testdata.508089 2026-02-17T12:33:32.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj889 testdata.508089 2026-02-17T12:33:32.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj890 testdata.508089 2026-02-17T12:33:32.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj891 testdata.508089 2026-02-17T12:33:32.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj892 testdata.508089 2026-02-17T12:33:32.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj893 testdata.508089 2026-02-17T12:33:32.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj894 testdata.508089 2026-02-17T12:33:32.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj895 testdata.508089 2026-02-17T12:33:32.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj896 testdata.508089 2026-02-17T12:33:32.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj897 testdata.508089 2026-02-17T12:33:32.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj898 testdata.508089 2026-02-17T12:33:32.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj899 testdata.508089 2026-02-17T12:33:32.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj900 testdata.508089 2026-02-17T12:33:32.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj901 testdata.508089 2026-02-17T12:33:32.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj902 testdata.508089 2026-02-17T12:33:32.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj903 testdata.508089 2026-02-17T12:33:32.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.744 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj904 testdata.508089 2026-02-17T12:33:32.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj905 testdata.508089 2026-02-17T12:33:32.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj906 testdata.508089 2026-02-17T12:33:32.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.810 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj907 testdata.508089 2026-02-17T12:33:32.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj908 testdata.508089 2026-02-17T12:33:32.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj909 testdata.508089 2026-02-17T12:33:32.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj910 testdata.508089 2026-02-17T12:33:32.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj911 testdata.508089 2026-02-17T12:33:32.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj912 testdata.508089 2026-02-17T12:33:32.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj913 testdata.508089 2026-02-17T12:33:32.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj914 testdata.508089 2026-02-17T12:33:32.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:32.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj915 testdata.508089 2026-02-17T12:33:33.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj916 testdata.508089 2026-02-17T12:33:33.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj917 testdata.508089 2026-02-17T12:33:33.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj918 testdata.508089 2026-02-17T12:33:33.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj919 testdata.508089 2026-02-17T12:33:33.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj920 testdata.508089 2026-02-17T12:33:33.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj921 testdata.508089 2026-02-17T12:33:33.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj922 testdata.508089 2026-02-17T12:33:33.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj923 testdata.508089 2026-02-17T12:33:33.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj924 testdata.508089 2026-02-17T12:33:33.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj925 testdata.508089 2026-02-17T12:33:33.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj926 testdata.508089 2026-02-17T12:33:33.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.280 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj927 testdata.508089 2026-02-17T12:33:33.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj928 testdata.508089 2026-02-17T12:33:33.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj929 testdata.508089 2026-02-17T12:33:33.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj930 testdata.508089 2026-02-17T12:33:33.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.368 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj931 testdata.508089 2026-02-17T12:33:33.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj932 testdata.508089 2026-02-17T12:33:33.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj933 testdata.508089 2026-02-17T12:33:33.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj934 testdata.508089 2026-02-17T12:33:33.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj935 testdata.508089 2026-02-17T12:33:33.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj936 testdata.508089 2026-02-17T12:33:33.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj937 testdata.508089 2026-02-17T12:33:33.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj938 testdata.508089 2026-02-17T12:33:33.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj939 testdata.508089 2026-02-17T12:33:33.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj940 testdata.508089 2026-02-17T12:33:33.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj941 testdata.508089 2026-02-17T12:33:33.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj942 testdata.508089 2026-02-17T12:33:33.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj943 testdata.508089 2026-02-17T12:33:33.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.675 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj944 testdata.508089 2026-02-17T12:33:33.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj945 testdata.508089 2026-02-17T12:33:33.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj946 testdata.508089 2026-02-17T12:33:33.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj947 testdata.508089 2026-02-17T12:33:33.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj948 testdata.508089 2026-02-17T12:33:33.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.798 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj949 testdata.508089 2026-02-17T12:33:33.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj950 testdata.508089 2026-02-17T12:33:33.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.846 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj951 testdata.508089 2026-02-17T12:33:33.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj952 testdata.508089 2026-02-17T12:33:33.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj953 testdata.508089 2026-02-17T12:33:33.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj954 testdata.508089 2026-02-17T12:33:33.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj955 testdata.508089 2026-02-17T12:33:33.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj956 testdata.508089 2026-02-17T12:33:33.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:33.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj957 testdata.508089 2026-02-17T12:33:34.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj958 testdata.508089 2026-02-17T12:33:34.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj959 testdata.508089 2026-02-17T12:33:34.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj960 testdata.508089 2026-02-17T12:33:34.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj961 testdata.508089 2026-02-17T12:33:34.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj962 testdata.508089 2026-02-17T12:33:34.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.129 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj963 testdata.508089 2026-02-17T12:33:34.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj964 testdata.508089 2026-02-17T12:33:34.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj965 testdata.508089 2026-02-17T12:33:34.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj966 testdata.508089 2026-02-17T12:33:34.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj967 testdata.508089 2026-02-17T12:33:34.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj968 testdata.508089 2026-02-17T12:33:34.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj969 testdata.508089 2026-02-17T12:33:34.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj970 testdata.508089 2026-02-17T12:33:34.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj971 testdata.508089 2026-02-17T12:33:34.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj972 testdata.508089 2026-02-17T12:33:34.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.376 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj973 testdata.508089 2026-02-17T12:33:34.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj974 testdata.508089 2026-02-17T12:33:34.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj975 testdata.508089 2026-02-17T12:33:34.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj976 testdata.508089 2026-02-17T12:33:34.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj977 testdata.508089 2026-02-17T12:33:34.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj978 testdata.508089 2026-02-17T12:33:34.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj979 testdata.508089 2026-02-17T12:33:34.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj980 testdata.508089 2026-02-17T12:33:34.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj981 testdata.508089 2026-02-17T12:33:34.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj982 testdata.508089 2026-02-17T12:33:34.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj983 testdata.508089 2026-02-17T12:33:34.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj984 testdata.508089 2026-02-17T12:33:34.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj985 testdata.508089 2026-02-17T12:33:34.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.683 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj986 testdata.508089 2026-02-17T12:33:34.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.707 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj987 testdata.508089 2026-02-17T12:33:34.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj988 testdata.508089 2026-02-17T12:33:34.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj989 testdata.508089 2026-02-17T12:33:34.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj990 testdata.508089 2026-02-17T12:33:34.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj991 testdata.508089 2026-02-17T12:33:34.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj992 testdata.508089 2026-02-17T12:33:34.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj993 testdata.508089 2026-02-17T12:33:34.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj994 testdata.508089 2026-02-17T12:33:34.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj995 testdata.508089 2026-02-17T12:33:34.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.926 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj996 testdata.508089 2026-02-17T12:33:34.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj997 testdata.508089 2026-02-17T12:33:34.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.975 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj998 testdata.508089 2026-02-17T12:33:34.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:34.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj999 testdata.508089 2026-02-17T12:33:35.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:335: _scrub_abort: for i in `seq 1 $objects` 2026-02-17T12:33:35.024 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:337: _scrub_abort: rados -p test put obj1000 testdata.508089 2026-02-17T12:33:35.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:339: _scrub_abort: rm -f testdata.508089 2026-02-17T12:33:35.049 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:341: _scrub_abort: get_primary test obj1 2026-02-17T12:33:35.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T12:33:35.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T12:33:35.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T12:33:35.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:33:35.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:341: _scrub_abort: local primary=1 2026-02-17T12:33:35.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:342: _scrub_abort: local pgid=1.0 2026-02-17T12:33:35.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:344: _scrub_abort: ceph tell 1.0 schedule-scrub 2026-02-17T12:33:35.270 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:33:35.270 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:33:35.270 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:33:35.270 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-10T12:31:55.265558+0000" 2026-02-17T12:33:35.270 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:33:35.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:347: _scrub_abort: set -o pipefail 2026-02-17T12:33:35.277 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:348: _scrub_abort: found=no 2026-02-17T12:33:35.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:349: _scrub_abort: seq 0 200 2026-02-17T12:33:35.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:349: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:33:35.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:351: _scrub_abort: flush_pg_stats 2026-02-17T12:33:35.278 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:33:35.279 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:33:35.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:33:35.435 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:33:35.436 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:33:35.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:33:35.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:35.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:33:35.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836526 2026-02-17T12:33:35.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836526 2026-02-17T12:33:35.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836526' 2026-02-17T12:33:35.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:35.506 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:33:35.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672998 2026-02-17T12:33:35.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672998 2026-02-17T12:33:35.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836526 1-42949672998' 2026-02-17T12:33:35.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:35.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:33:35.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542175 2026-02-17T12:33:35.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542175 2026-02-17T12:33:35.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836526 1-42949672998 2-60129542175' 2026-02-17T12:33:35.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:35.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836526 2026-02-17T12:33:35.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:35.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:33:35.649 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836526 2026-02-17T12:33:35.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:35.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836526 2026-02-17T12:33:35.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836526' 2026-02-17T12:33:35.652 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836526 2026-02-17T12:33:35.652 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:35.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836526 -lt 21474836526 2026-02-17T12:33:35.805 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:35.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672998 2026-02-17T12:33:35.806 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:35.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:33:35.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672998 2026-02-17T12:33:35.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:35.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672998 2026-02-17T12:33:35.809 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672998' 2026-02-17T12:33:35.810 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672998 2026-02-17T12:33:35.810 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:33:35.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672998 -lt 42949672998 2026-02-17T12:33:35.972 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:35.973 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542175 2026-02-17T12:33:35.973 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:35.974 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:33:35.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542175 2026-02-17T12:33:35.975 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:35.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542175 2026-02-17T12:33:35.976 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542175' 2026-02-17T12:33:35.977 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542175 2026-02-17T12:33:35.977 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:33:36.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542174 -lt 60129542175 2026-02-17T12:33:36.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:33:37.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:33:37.139 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:33:37.287 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542174 -lt 60129542175 2026-02-17T12:33:37.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:33:38.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-17T12:33:38.290 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:33:38.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542177 -lt 60129542175 2026-02-17T12:33:38.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: ceph pg dump pgs 2026-02-17T12:33:38.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: grep '^1.0' 2026-02-17T12:33:38.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:352: _scrub_abort: grep -q scrubbing 2026-02-17T12:33:38.581 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:33:38.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:354: _scrub_abort: found=yes 2026-02-17T12:33:38.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:356: _scrub_abort: break 2026-02-17T12:33:38.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:359: _scrub_abort: set +o pipefail 2026-02-17T12:33:38.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:361: _scrub_abort: test yes = no 2026-02-17T12:33:38.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:367: _scrub_abort: ceph osd set noscrub 2026-02-17T12:33:38.788 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is set 2026-02-17T12:33:38.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:368: _scrub_abort: '[' scrub = deep-scrub ']' 2026-02-17T12:33:38.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:374: _scrub_abort: set -o pipefail 2026-02-17T12:33:38.800 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: seq 0 200 2026-02-17T12:33:38.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:33:38.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:33:38.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:33:38.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:33:38.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:33:38.949 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:33:38.949 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:33:38.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:33:38.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:38.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:33:39.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836530 2026-02-17T12:33:39.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836530 2026-02-17T12:33:39.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836530' 2026-02-17T12:33:39.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:39.021 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:33:39.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673003 2026-02-17T12:33:39.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673003 2026-02-17T12:33:39.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836530 1-42949673003' 2026-02-17T12:33:39.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:39.090 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:33:39.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542179 2026-02-17T12:33:39.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542179 2026-02-17T12:33:39.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836530 1-42949673003 2-60129542179' 2026-02-17T12:33:39.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:39.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836530 2026-02-17T12:33:39.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:39.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:33:39.162 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836530 2026-02-17T12:33:39.162 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:39.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836530 2026-02-17T12:33:39.163 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836530 2026-02-17T12:33:39.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836530' 2026-02-17T12:33:39.164 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:39.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836528 -lt 21474836530 2026-02-17T12:33:39.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:33:40.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:33:40.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:40.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836531 -lt 21474836530 2026-02-17T12:33:40.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:40.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673003 2026-02-17T12:33:40.476 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:40.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:33:40.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673003 2026-02-17T12:33:40.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:40.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673003 2026-02-17T12:33:40.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673003' 2026-02-17T12:33:40.479 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673003 2026-02-17T12:33:40.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:33:40.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673003 -lt 42949673003 2026-02-17T12:33:40.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:40.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542179 2026-02-17T12:33:40.631 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:40.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:33:40.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542179 2026-02-17T12:33:40.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:40.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542179 2026-02-17T12:33:40.634 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542179 2026-02-17T12:33:40.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542179' 2026-02-17T12:33:40.635 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:33:40.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542180 -lt 60129542179 2026-02-17T12:33:40.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:33:40.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:33:40.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:33:40.935 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:33:40.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:33:40.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:33:40.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:33:40.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:33:40.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:33:41.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:33:41.097 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:33:41.097 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:33:41.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:33:41.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:41.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:33:41.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836533 2026-02-17T12:33:41.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836533 2026-02-17T12:33:41.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533' 2026-02-17T12:33:41.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:41.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:33:41.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673006 2026-02-17T12:33:41.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673006 2026-02-17T12:33:41.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-42949673006' 2026-02-17T12:33:41.240 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:41.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:33:41.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542182 2026-02-17T12:33:41.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542182 2026-02-17T12:33:41.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-42949673006 2-60129542182' 2026-02-17T12:33:41.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:41.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836533 2026-02-17T12:33:41.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:41.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:33:41.311 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836533 2026-02-17T12:33:41.311 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:41.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836533 2026-02-17T12:33:41.313 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836533 2026-02-17T12:33:41.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836533' 2026-02-17T12:33:41.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:41.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836531 -lt 21474836533 2026-02-17T12:33:41.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:33:42.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:33:42.476 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:42.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836534 -lt 21474836533 2026-02-17T12:33:42.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:42.625 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673006 2026-02-17T12:33:42.625 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:42.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:33:42.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673006 2026-02-17T12:33:42.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:42.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673006 2026-02-17T12:33:42.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673006' 2026-02-17T12:33:42.628 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673006 2026-02-17T12:33:42.629 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:33:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673006 -lt 42949673006 2026-02-17T12:33:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:42.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542182 2026-02-17T12:33:42.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:42.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:33:42.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542182 2026-02-17T12:33:42.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:42.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542182 2026-02-17T12:33:42.796 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542182 2026-02-17T12:33:42.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542182' 2026-02-17T12:33:42.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:33:42.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542183 -lt 60129542182 2026-02-17T12:33:42.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:33:42.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:33:42.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:33:43.090 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:33:43.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:33:43.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:33:43.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:33:43.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:33:43.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:33:43.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:33:43.254 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:33:43.254 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:33:43.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:33:43.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:43.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:33:43.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836537 2026-02-17T12:33:43.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836537 2026-02-17T12:33:43.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836537' 2026-02-17T12:33:43.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:43.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:33:43.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673009 2026-02-17T12:33:43.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673009 2026-02-17T12:33:43.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836537 1-42949673009' 2026-02-17T12:33:43.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:43.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:33:43.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542186 2026-02-17T12:33:43.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542186 2026-02-17T12:33:43.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836537 1-42949673009 2-60129542186' 2026-02-17T12:33:43.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:43.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836537 2026-02-17T12:33:43.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:43.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:33:43.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836537 2026-02-17T12:33:43.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:43.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836537 2026-02-17T12:33:43.471 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836537 2026-02-17T12:33:43.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836537' 2026-02-17T12:33:43.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:43.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836534 -lt 21474836537 2026-02-17T12:33:43.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:33:44.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:33:44.629 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:44.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836537 -lt 21474836537 2026-02-17T12:33:44.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:44.780 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673009 2026-02-17T12:33:44.780 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:44.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:33:44.782 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673009 2026-02-17T12:33:44.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:44.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673009 2026-02-17T12:33:44.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673009' 2026-02-17T12:33:44.784 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673009 2026-02-17T12:33:44.784 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:33:44.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673009 -lt 42949673009 2026-02-17T12:33:44.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:44.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542186 2026-02-17T12:33:44.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:44.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:33:44.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542186 2026-02-17T12:33:44.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:44.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542186 2026-02-17T12:33:44.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542186' 2026-02-17T12:33:44.941 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542186 2026-02-17T12:33:44.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:33:45.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542186 -lt 60129542186 2026-02-17T12:33:45.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:33:45.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:33:45.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:33:45.237 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:33:45.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:33:45.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:33:45.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:33:45.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:33:45.247 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:33:45.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:33:45.392 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:33:45.392 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:33:45.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:33:45.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:45.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:33:45.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836540 2026-02-17T12:33:45.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836540 2026-02-17T12:33:45.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836540' 2026-02-17T12:33:45.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:45.464 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:33:45.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673012 2026-02-17T12:33:45.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673012 2026-02-17T12:33:45.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836540 1-42949673012' 2026-02-17T12:33:45.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:45.536 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:33:45.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542189 2026-02-17T12:33:45.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542189 2026-02-17T12:33:45.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836540 1-42949673012 2-60129542189' 2026-02-17T12:33:45.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:45.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836540 2026-02-17T12:33:45.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:45.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:33:45.611 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836540 2026-02-17T12:33:45.611 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:45.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836540 2026-02-17T12:33:45.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836540' 2026-02-17T12:33:45.613 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836540 2026-02-17T12:33:45.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:45.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836540 -lt 21474836540 2026-02-17T12:33:45.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:45.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673012 2026-02-17T12:33:45.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:45.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:33:45.774 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673012 2026-02-17T12:33:45.774 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:45.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673012 2026-02-17T12:33:45.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673012' 2026-02-17T12:33:45.776 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673012 2026-02-17T12:33:45.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:33:45.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673012 -lt 42949673012 2026-02-17T12:33:45.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:45.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542189 2026-02-17T12:33:45.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:45.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:33:45.942 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542189 2026-02-17T12:33:45.942 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:45.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542189 2026-02-17T12:33:45.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542189' 2026-02-17T12:33:45.944 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542189 2026-02-17T12:33:45.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:33:46.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542189 -lt 60129542189 2026-02-17T12:33:46.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:33:46.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:33:46.100 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:33:46.246 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:33:46.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:33:46.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:33:46.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:33:46.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:33:46.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:33:46.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:33:46.413 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:33:46.414 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:33:46.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:33:46.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:46.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:33:46.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836542 2026-02-17T12:33:46.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836542 2026-02-17T12:33:46.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836542' 2026-02-17T12:33:46.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:46.483 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:33:46.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673014 2026-02-17T12:33:46.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673014 2026-02-17T12:33:46.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836542 1-42949673014' 2026-02-17T12:33:46.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:46.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:33:46.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542191 2026-02-17T12:33:46.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542191 2026-02-17T12:33:46.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836542 1-42949673014 2-60129542191' 2026-02-17T12:33:46.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:46.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836542 2026-02-17T12:33:46.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:46.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:33:46.627 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836542 2026-02-17T12:33:46.627 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:46.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836542 2026-02-17T12:33:46.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836542' 2026-02-17T12:33:46.629 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836542 2026-02-17T12:33:46.629 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:46.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836540 -lt 21474836542 2026-02-17T12:33:46.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:33:47.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:33:47.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:47.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836543 -lt 21474836542 2026-02-17T12:33:47.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:47.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673014 2026-02-17T12:33:47.946 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:47.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:33:47.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673014 2026-02-17T12:33:47.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:47.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673014 2026-02-17T12:33:47.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673014' 2026-02-17T12:33:47.949 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673014 2026-02-17T12:33:47.950 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:33:48.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673015 -lt 42949673014 2026-02-17T12:33:48.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:48.113 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542191 2026-02-17T12:33:48.113 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:48.114 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:33:48.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542191 2026-02-17T12:33:48.115 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:48.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542191 2026-02-17T12:33:48.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542191' 2026-02-17T12:33:48.117 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542191 2026-02-17T12:33:48.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:33:48.273 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542192 -lt 60129542191 2026-02-17T12:33:48.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:33:48.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:33:48.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:33:48.416 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:33:48.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:33:48.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:33:48.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:33:48.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:33:48.427 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:33:48.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:33:48.579 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:33:48.579 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:33:48.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:33:48.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:48.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:33:48.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836545 2026-02-17T12:33:48.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836545 2026-02-17T12:33:48.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836545' 2026-02-17T12:33:48.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:48.649 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:33:48.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673017 2026-02-17T12:33:48.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673017 2026-02-17T12:33:48.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836545 1-42949673017' 2026-02-17T12:33:48.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:48.720 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:33:48.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542194 2026-02-17T12:33:48.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542194 2026-02-17T12:33:48.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836545 1-42949673017 2-60129542194' 2026-02-17T12:33:48.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:48.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836545 2026-02-17T12:33:48.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:48.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:33:48.793 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836545 2026-02-17T12:33:48.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:48.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836545 2026-02-17T12:33:48.795 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836545 2026-02-17T12:33:48.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836545' 2026-02-17T12:33:48.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:48.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836543 -lt 21474836545 2026-02-17T12:33:48.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:33:49.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:33:49.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:50.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836546 -lt 21474836545 2026-02-17T12:33:50.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:50.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673017 2026-02-17T12:33:50.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:50.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:33:50.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673017 2026-02-17T12:33:50.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:50.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673017 2026-02-17T12:33:50.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673017' 2026-02-17T12:33:50.106 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673017 2026-02-17T12:33:50.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:33:50.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673018 -lt 42949673017 2026-02-17T12:33:50.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:50.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542194 2026-02-17T12:33:50.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:50.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:33:50.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542194 2026-02-17T12:33:50.266 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:50.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542194 2026-02-17T12:33:50.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542194' 2026-02-17T12:33:50.267 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542194 2026-02-17T12:33:50.267 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:33:50.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542195 -lt 60129542194 2026-02-17T12:33:50.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:33:50.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:33:50.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:33:50.569 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:33:50.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:33:50.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:33:50.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:33:50.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:33:50.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:33:50.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:33:50.725 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:33:50.725 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:33:50.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:33:50.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:50.726 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:33:50.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836548 2026-02-17T12:33:50.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836548 2026-02-17T12:33:50.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836548' 2026-02-17T12:33:50.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:50.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:33:50.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673021 2026-02-17T12:33:50.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673021 2026-02-17T12:33:50.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836548 1-42949673021' 2026-02-17T12:33:50.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:50.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:33:50.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542197 2026-02-17T12:33:50.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542197 2026-02-17T12:33:50.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836548 1-42949673021 2-60129542197' 2026-02-17T12:33:50.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:50.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836548 2026-02-17T12:33:50.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:50.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:33:50.942 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836548 2026-02-17T12:33:50.942 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:50.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836548 2026-02-17T12:33:50.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836548' 2026-02-17T12:33:50.944 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836548 2026-02-17T12:33:50.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:51.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836546 -lt 21474836548 2026-02-17T12:33:51.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:33:52.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:33:52.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:52.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836549 -lt 21474836548 2026-02-17T12:33:52.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:52.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673021 2026-02-17T12:33:52.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:52.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:33:52.251 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673021 2026-02-17T12:33:52.251 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:52.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673021 2026-02-17T12:33:52.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673021' 2026-02-17T12:33:52.253 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673021 2026-02-17T12:33:52.253 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:33:52.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673021 -lt 42949673021 2026-02-17T12:33:52.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:52.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542197 2026-02-17T12:33:52.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:52.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:33:52.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542197 2026-02-17T12:33:52.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:52.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542197 2026-02-17T12:33:52.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542197' 2026-02-17T12:33:52.418 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542197 2026-02-17T12:33:52.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:33:52.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542198 -lt 60129542197 2026-02-17T12:33:52.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:33:52.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:33:52.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:33:52.724 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:33:52.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:33:52.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:33:52.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:33:52.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:33:52.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:33:52.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:33:52.886 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:33:52.886 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:33:52.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:33:52.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:52.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:33:52.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836551 2026-02-17T12:33:52.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836551 2026-02-17T12:33:52.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836551' 2026-02-17T12:33:52.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:52.956 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:33:53.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673024 2026-02-17T12:33:53.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673024 2026-02-17T12:33:53.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836551 1-42949673024' 2026-02-17T12:33:53.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:53.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:33:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542200 2026-02-17T12:33:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542200 2026-02-17T12:33:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836551 1-42949673024 2-60129542200' 2026-02-17T12:33:53.099 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:53.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836551 2026-02-17T12:33:53.100 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:53.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:33:53.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836551 2026-02-17T12:33:53.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:53.103 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836551 2026-02-17T12:33:53.103 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836551 2026-02-17T12:33:53.104 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836551' 2026-02-17T12:33:53.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:53.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836549 -lt 21474836551 2026-02-17T12:33:53.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:33:54.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:33:54.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:54.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836552 -lt 21474836551 2026-02-17T12:33:54.418 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:54.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673024 2026-02-17T12:33:54.419 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:54.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:33:54.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673024 2026-02-17T12:33:54.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:54.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673024 2026-02-17T12:33:54.422 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673024 2026-02-17T12:33:54.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673024' 2026-02-17T12:33:54.422 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:33:54.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673024 -lt 42949673024 2026-02-17T12:33:54.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:54.584 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542200 2026-02-17T12:33:54.584 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:54.585 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:33:54.585 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542200 2026-02-17T12:33:54.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:54.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542200 2026-02-17T12:33:54.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542200' 2026-02-17T12:33:54.588 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542200 2026-02-17T12:33:54.588 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:33:54.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542201 -lt 60129542200 2026-02-17T12:33:54.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:33:54.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:33:54.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:33:54.880 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:33:54.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:33:54.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:33:54.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:33:54.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:33:54.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:33:55.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:33:55.036 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:33:55.037 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:33:55.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:33:55.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:55.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:33:55.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836554 2026-02-17T12:33:55.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836554 2026-02-17T12:33:55.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836554' 2026-02-17T12:33:55.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:55.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:33:55.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673027 2026-02-17T12:33:55.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673027 2026-02-17T12:33:55.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836554 1-42949673027' 2026-02-17T12:33:55.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:55.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:33:55.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542203 2026-02-17T12:33:55.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542203 2026-02-17T12:33:55.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836554 1-42949673027 2-60129542203' 2026-02-17T12:33:55.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:55.253 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836554 2026-02-17T12:33:55.253 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:55.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:33:55.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836554 2026-02-17T12:33:55.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:55.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836554 2026-02-17T12:33:55.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836554' 2026-02-17T12:33:55.256 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836554 2026-02-17T12:33:55.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:55.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836552 -lt 21474836554 2026-02-17T12:33:55.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:33:56.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:33:56.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:56.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836555 -lt 21474836554 2026-02-17T12:33:56.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:56.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673027 2026-02-17T12:33:56.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:56.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:33:56.566 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673027 2026-02-17T12:33:56.566 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:56.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673027 2026-02-17T12:33:56.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673027' 2026-02-17T12:33:56.568 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673027 2026-02-17T12:33:56.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:33:56.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673027 -lt 42949673027 2026-02-17T12:33:56.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:56.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542203 2026-02-17T12:33:56.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:56.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:33:56.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542203 2026-02-17T12:33:56.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:56.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542203 2026-02-17T12:33:56.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542203' 2026-02-17T12:33:56.731 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542203 2026-02-17T12:33:56.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:33:56.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542204 -lt 60129542203 2026-02-17T12:33:56.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:33:56.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:33:56.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:33:57.027 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:33:57.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:33:57.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:33:57.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:33:57.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:33:57.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:33:57.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:33:57.185 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:33:57.185 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:33:57.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:33:57.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:57.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:33:57.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836558 2026-02-17T12:33:57.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836558 2026-02-17T12:33:57.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836558' 2026-02-17T12:33:57.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:57.257 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:33:57.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673030 2026-02-17T12:33:57.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673030 2026-02-17T12:33:57.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836558 1-42949673030' 2026-02-17T12:33:57.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:57.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:33:57.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542207 2026-02-17T12:33:57.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542207 2026-02-17T12:33:57.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836558 1-42949673030 2-60129542207' 2026-02-17T12:33:57.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:57.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836558 2026-02-17T12:33:57.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:57.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:33:57.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836558 2026-02-17T12:33:57.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:57.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836558 2026-02-17T12:33:57.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836558' 2026-02-17T12:33:57.404 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836558 2026-02-17T12:33:57.404 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:57.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836555 -lt 21474836558 2026-02-17T12:33:57.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:33:58.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:33:58.566 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:58.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836558 -lt 21474836558 2026-02-17T12:33:58.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:58.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673030 2026-02-17T12:33:58.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:58.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:33:58.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673030 2026-02-17T12:33:58.722 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:58.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673030 2026-02-17T12:33:58.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673030' 2026-02-17T12:33:58.723 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673030 2026-02-17T12:33:58.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:33:58.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673030 -lt 42949673030 2026-02-17T12:33:58.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:58.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542207 2026-02-17T12:33:58.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:58.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:33:58.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542207 2026-02-17T12:33:58.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:58.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542207 2026-02-17T12:33:58.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542207' 2026-02-17T12:33:58.890 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542207 2026-02-17T12:33:58.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:33:59.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542207 -lt 60129542207 2026-02-17T12:33:59.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:33:59.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:33:59.057 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:33:59.195 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:33:59.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:33:59.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:33:59.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:33:59.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:33:59.205 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:33:59.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:33:59.352 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:33:59.352 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:33:59.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:33:59.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:59.353 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:33:59.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836561 2026-02-17T12:33:59.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836561 2026-02-17T12:33:59.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836561' 2026-02-17T12:33:59.426 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:59.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:33:59.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673033 2026-02-17T12:33:59.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673033 2026-02-17T12:33:59.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836561 1-42949673033' 2026-02-17T12:33:59.496 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:33:59.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:33:59.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542210 2026-02-17T12:33:59.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542210 2026-02-17T12:33:59.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836561 1-42949673033 2-60129542210' 2026-02-17T12:33:59.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:59.567 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836561 2026-02-17T12:33:59.567 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:59.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:33:59.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836561 2026-02-17T12:33:59.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:59.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836561 2026-02-17T12:33:59.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836561' 2026-02-17T12:33:59.571 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836561 2026-02-17T12:33:59.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:33:59.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836561 -lt 21474836561 2026-02-17T12:33:59.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:59.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673033 2026-02-17T12:33:59.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:59.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:33:59.726 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673033 2026-02-17T12:33:59.726 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:59.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673033 2026-02-17T12:33:59.728 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673033 2026-02-17T12:33:59.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673033' 2026-02-17T12:33:59.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:33:59.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673033 -lt 42949673033 2026-02-17T12:33:59.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:33:59.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542210 2026-02-17T12:33:59.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:33:59.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:33:59.892 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542210 2026-02-17T12:33:59.892 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:33:59.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542210 2026-02-17T12:33:59.893 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542210 2026-02-17T12:33:59.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542210' 2026-02-17T12:33:59.894 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:00.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542210 -lt 60129542210 2026-02-17T12:34:00.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:00.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:00.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:00.197 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:00.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:00.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:00.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:00.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:00.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:00.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:00.361 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:00.361 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:00.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:00.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:00.362 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:00.431 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836563 2026-02-17T12:34:00.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836563 2026-02-17T12:34:00.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836563' 2026-02-17T12:34:00.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:00.432 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:00.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673035 2026-02-17T12:34:00.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673035 2026-02-17T12:34:00.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836563 1-42949673035' 2026-02-17T12:34:00.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:00.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:00.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542212 2026-02-17T12:34:00.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542212 2026-02-17T12:34:00.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836563 1-42949673035 2-60129542212' 2026-02-17T12:34:00.575 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:00.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836563 2026-02-17T12:34:00.575 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:00.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:00.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836563 2026-02-17T12:34:00.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:00.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836563 2026-02-17T12:34:00.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836563' 2026-02-17T12:34:00.579 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836563 2026-02-17T12:34:00.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:00.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836561 -lt 21474836563 2026-02-17T12:34:00.731 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:01.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:01.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:01.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836564 -lt 21474836563 2026-02-17T12:34:01.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:01.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673035 2026-02-17T12:34:01.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:01.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:01.883 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673035 2026-02-17T12:34:01.884 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:01.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673035 2026-02-17T12:34:01.885 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673035 2026-02-17T12:34:01.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673035' 2026-02-17T12:34:01.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:02.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673036 -lt 42949673035 2026-02-17T12:34:02.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:02.046 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542212 2026-02-17T12:34:02.046 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:02.047 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:02.048 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542212 2026-02-17T12:34:02.049 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:02.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542212 2026-02-17T12:34:02.050 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542212 2026-02-17T12:34:02.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542212' 2026-02-17T12:34:02.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:02.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542213 -lt 60129542212 2026-02-17T12:34:02.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:02.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:02.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:02.350 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:02.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:02.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:02.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:02.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:02.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:02.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:02.509 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:02.509 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:02.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:02.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:02.509 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:02.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836566 2026-02-17T12:34:02.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836566 2026-02-17T12:34:02.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836566' 2026-02-17T12:34:02.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:02.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:02.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673038 2026-02-17T12:34:02.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673038 2026-02-17T12:34:02.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836566 1-42949673038' 2026-02-17T12:34:02.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:02.653 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:02.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542215 2026-02-17T12:34:02.723 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542215 2026-02-17T12:34:02.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836566 1-42949673038 2-60129542215' 2026-02-17T12:34:02.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:02.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836566 2026-02-17T12:34:02.724 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:02.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:02.726 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836566 2026-02-17T12:34:02.726 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:02.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836566 2026-02-17T12:34:02.727 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836566 2026-02-17T12:34:02.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836566' 2026-02-17T12:34:02.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:02.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836564 -lt 21474836566 2026-02-17T12:34:02.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:03.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:03.875 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:04.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836567 -lt 21474836566 2026-02-17T12:34:04.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:04.030 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673038 2026-02-17T12:34:04.030 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:04.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:04.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673038 2026-02-17T12:34:04.032 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:04.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673038 2026-02-17T12:34:04.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673038' 2026-02-17T12:34:04.034 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673038 2026-02-17T12:34:04.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:04.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673039 -lt 42949673038 2026-02-17T12:34:04.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:04.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542215 2026-02-17T12:34:04.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:04.193 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:04.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542215 2026-02-17T12:34:04.194 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:04.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542215 2026-02-17T12:34:04.196 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542215 2026-02-17T12:34:04.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542215' 2026-02-17T12:34:04.196 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:04.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542216 -lt 60129542215 2026-02-17T12:34:04.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:04.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:04.352 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:04.493 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:04.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:04.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:04.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:04.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:04.502 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:04.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:04.647 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:04.647 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:04.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:04.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:04.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:04.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836569 2026-02-17T12:34:04.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836569 2026-02-17T12:34:04.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836569' 2026-02-17T12:34:04.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:04.720 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:04.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673042 2026-02-17T12:34:04.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673042 2026-02-17T12:34:04.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836569 1-42949673042' 2026-02-17T12:34:04.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:04.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:04.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542218 2026-02-17T12:34:04.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542218 2026-02-17T12:34:04.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836569 1-42949673042 2-60129542218' 2026-02-17T12:34:04.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:04.861 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836569 2026-02-17T12:34:04.862 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:04.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:04.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836569 2026-02-17T12:34:04.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:04.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836569 2026-02-17T12:34:04.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836569' 2026-02-17T12:34:04.865 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836569 2026-02-17T12:34:04.865 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:05.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836567 -lt 21474836569 2026-02-17T12:34:05.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:06.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:06.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:06.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836570 -lt 21474836569 2026-02-17T12:34:06.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:06.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673042 2026-02-17T12:34:06.180 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:06.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:06.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673042 2026-02-17T12:34:06.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:06.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673042 2026-02-17T12:34:06.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673042' 2026-02-17T12:34:06.183 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673042 2026-02-17T12:34:06.184 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:06.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673042 -lt 42949673042 2026-02-17T12:34:06.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:06.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542218 2026-02-17T12:34:06.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:06.346 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:06.347 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542218 2026-02-17T12:34:06.347 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:06.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542218 2026-02-17T12:34:06.348 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542218 2026-02-17T12:34:06.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542218' 2026-02-17T12:34:06.349 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:06.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542219 -lt 60129542218 2026-02-17T12:34:06.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:06.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:06.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:06.642 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:06.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:06.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:06.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:06.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:06.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:06.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:06.800 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:06.800 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:06.800 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:06.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:06.801 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:06.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836572 2026-02-17T12:34:06.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836572 2026-02-17T12:34:06.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836572' 2026-02-17T12:34:06.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:06.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:06.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673045 2026-02-17T12:34:06.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673045 2026-02-17T12:34:06.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836572 1-42949673045' 2026-02-17T12:34:06.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:06.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:07.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542221 2026-02-17T12:34:07.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542221 2026-02-17T12:34:07.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836572 1-42949673045 2-60129542221' 2026-02-17T12:34:07.012 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:07.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836572 2026-02-17T12:34:07.013 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:07.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:07.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836572 2026-02-17T12:34:07.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:07.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836572 2026-02-17T12:34:07.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836572' 2026-02-17T12:34:07.017 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836572 2026-02-17T12:34:07.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:07.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836570 -lt 21474836572 2026-02-17T12:34:07.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:08.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:08.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:08.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836573 -lt 21474836572 2026-02-17T12:34:08.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:08.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673045 2026-02-17T12:34:08.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:08.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:08.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673045 2026-02-17T12:34:08.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:08.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673045 2026-02-17T12:34:08.337 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673045 2026-02-17T12:34:08.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673045' 2026-02-17T12:34:08.338 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:08.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673045 -lt 42949673045 2026-02-17T12:34:08.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:08.498 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542221 2026-02-17T12:34:08.498 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:08.499 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:08.500 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542221 2026-02-17T12:34:08.500 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:08.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542221 2026-02-17T12:34:08.501 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542221 2026-02-17T12:34:08.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542221' 2026-02-17T12:34:08.502 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:08.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542222 -lt 60129542221 2026-02-17T12:34:08.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:08.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:08.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:08.810 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:08.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:08.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:08.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:08.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:08.820 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:08.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:08.968 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:08.968 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:08.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:08.968 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:08.968 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:09.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836575 2026-02-17T12:34:09.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836575 2026-02-17T12:34:09.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836575' 2026-02-17T12:34:09.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:09.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:09.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673048 2026-02-17T12:34:09.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673048 2026-02-17T12:34:09.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836575 1-42949673048' 2026-02-17T12:34:09.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:09.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:09.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542224 2026-02-17T12:34:09.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542224 2026-02-17T12:34:09.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836575 1-42949673048 2-60129542224' 2026-02-17T12:34:09.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:09.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836575 2026-02-17T12:34:09.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:09.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:09.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836575 2026-02-17T12:34:09.182 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:09.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836575 2026-02-17T12:34:09.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836575' 2026-02-17T12:34:09.184 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836575 2026-02-17T12:34:09.184 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:09.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836573 -lt 21474836575 2026-02-17T12:34:09.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:10.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:10.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:10.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836576 -lt 21474836575 2026-02-17T12:34:10.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:10.493 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673048 2026-02-17T12:34:10.493 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:10.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:10.495 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673048 2026-02-17T12:34:10.495 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:10.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673048 2026-02-17T12:34:10.497 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673048' 2026-02-17T12:34:10.497 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673048 2026-02-17T12:34:10.497 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:10.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673048 -lt 42949673048 2026-02-17T12:34:10.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:10.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542224 2026-02-17T12:34:10.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:10.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:10.661 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542224 2026-02-17T12:34:10.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:10.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542224 2026-02-17T12:34:10.663 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542224 2026-02-17T12:34:10.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542224' 2026-02-17T12:34:10.664 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:10.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542225 -lt 60129542224 2026-02-17T12:34:10.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:10.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:10.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:10.960 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:10.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:10.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:10.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:10.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:10.970 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:11.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:11.123 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:11.123 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:11.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:11.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:11.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:11.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836578 2026-02-17T12:34:11.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836578 2026-02-17T12:34:11.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836578' 2026-02-17T12:34:11.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:11.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:11.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673051 2026-02-17T12:34:11.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673051 2026-02-17T12:34:11.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836578 1-42949673051' 2026-02-17T12:34:11.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:11.264 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:11.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542228 2026-02-17T12:34:11.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542228 2026-02-17T12:34:11.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836578 1-42949673051 2-60129542228' 2026-02-17T12:34:11.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:11.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836578 2026-02-17T12:34:11.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:11.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:11.338 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836578 2026-02-17T12:34:11.338 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:11.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836578 2026-02-17T12:34:11.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836578' 2026-02-17T12:34:11.340 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836578 2026-02-17T12:34:11.340 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:11.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836576 -lt 21474836578 2026-02-17T12:34:11.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:12.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:12.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:12.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836579 -lt 21474836578 2026-02-17T12:34:12.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:12.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673051 2026-02-17T12:34:12.654 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:12.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:12.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673051 2026-02-17T12:34:12.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:12.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673051 2026-02-17T12:34:12.658 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673051 2026-02-17T12:34:12.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673051' 2026-02-17T12:34:12.658 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:12.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673051 -lt 42949673051 2026-02-17T12:34:12.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:12.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542228 2026-02-17T12:34:12.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:12.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:12.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542228 2026-02-17T12:34:12.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:12.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542228 2026-02-17T12:34:12.828 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542228 2026-02-17T12:34:12.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542228' 2026-02-17T12:34:12.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:12.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542228 -lt 60129542228 2026-02-17T12:34:12.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:12.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:12.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:13.126 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:13.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:13.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:13.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:13.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:13.138 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:13.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:13.292 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:13.292 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:13.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:13.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:13.292 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:13.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836582 2026-02-17T12:34:13.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836582 2026-02-17T12:34:13.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836582' 2026-02-17T12:34:13.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:13.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:13.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673054 2026-02-17T12:34:13.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673054 2026-02-17T12:34:13.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836582 1-42949673054' 2026-02-17T12:34:13.433 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:13.433 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:13.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542231 2026-02-17T12:34:13.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542231 2026-02-17T12:34:13.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836582 1-42949673054 2-60129542231' 2026-02-17T12:34:13.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:13.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836582 2026-02-17T12:34:13.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:13.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:13.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836582 2026-02-17T12:34:13.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:13.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836582 2026-02-17T12:34:13.507 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836582 2026-02-17T12:34:13.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836582' 2026-02-17T12:34:13.507 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:13.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836579 -lt 21474836582 2026-02-17T12:34:13.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:14.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:14.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:14.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836582 -lt 21474836582 2026-02-17T12:34:14.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:14.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673054 2026-02-17T12:34:14.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:14.824 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:14.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673054 2026-02-17T12:34:14.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:14.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673054 2026-02-17T12:34:14.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673054' 2026-02-17T12:34:14.827 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673054 2026-02-17T12:34:14.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:14.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673054 -lt 42949673054 2026-02-17T12:34:14.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:14.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542231 2026-02-17T12:34:14.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:14.985 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:14.985 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542231 2026-02-17T12:34:14.986 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:14.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542231 2026-02-17T12:34:14.987 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542231 2026-02-17T12:34:14.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542231' 2026-02-17T12:34:14.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:15.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542231 -lt 60129542231 2026-02-17T12:34:15.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:15.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:15.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:15.279 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:15.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:15.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:15.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:15.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:15.289 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:15.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:15.441 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:15.441 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:15.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:15.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:15.442 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:15.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836585 2026-02-17T12:34:15.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836585 2026-02-17T12:34:15.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836585' 2026-02-17T12:34:15.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:15.512 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:15.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673057 2026-02-17T12:34:15.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673057 2026-02-17T12:34:15.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836585 1-42949673057' 2026-02-17T12:34:15.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:15.586 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:15.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542234 2026-02-17T12:34:15.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542234 2026-02-17T12:34:15.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836585 1-42949673057 2-60129542234' 2026-02-17T12:34:15.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:15.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836585 2026-02-17T12:34:15.658 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:15.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:15.659 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836585 2026-02-17T12:34:15.659 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:15.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836585 2026-02-17T12:34:15.661 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836585 2026-02-17T12:34:15.661 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836585' 2026-02-17T12:34:15.661 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:15.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836585 -lt 21474836585 2026-02-17T12:34:15.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:15.818 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673057 2026-02-17T12:34:15.818 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:15.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:15.820 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673057 2026-02-17T12:34:15.820 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:15.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673057 2026-02-17T12:34:15.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673057' 2026-02-17T12:34:15.821 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673057 2026-02-17T12:34:15.822 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:15.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673057 -lt 42949673057 2026-02-17T12:34:15.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:15.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542234 2026-02-17T12:34:15.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:15.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:15.990 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542234 2026-02-17T12:34:15.990 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:15.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542234 2026-02-17T12:34:15.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542234' 2026-02-17T12:34:15.992 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542234 2026-02-17T12:34:15.992 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:16.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542233 -lt 60129542234 2026-02-17T12:34:16.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:17.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:17.145 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:17.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542233 -lt 60129542234 2026-02-17T12:34:17.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:18.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-17T12:34:18.303 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:18.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542236 -lt 60129542234 2026-02-17T12:34:18.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:18.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:18.458 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:18.600 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:18.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:18.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:18.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:18.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:18.610 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:18.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:18.764 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:18.764 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:18.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:18.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:18.765 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:18.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836589 2026-02-17T12:34:18.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836589 2026-02-17T12:34:18.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836589' 2026-02-17T12:34:18.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:18.835 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:18.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673062 2026-02-17T12:34:18.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673062 2026-02-17T12:34:18.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836589 1-42949673062' 2026-02-17T12:34:18.908 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:18.908 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:18.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542238 2026-02-17T12:34:18.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542238 2026-02-17T12:34:18.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836589 1-42949673062 2-60129542238' 2026-02-17T12:34:18.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:18.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836589 2026-02-17T12:34:18.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:18.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:18.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836589 2026-02-17T12:34:18.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:18.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836589 2026-02-17T12:34:18.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836589' 2026-02-17T12:34:18.984 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836589 2026-02-17T12:34:18.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:19.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836587 -lt 21474836589 2026-02-17T12:34:19.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:20.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:20.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:20.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836590 -lt 21474836589 2026-02-17T12:34:20.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:20.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673062 2026-02-17T12:34:20.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:20.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:20.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673062 2026-02-17T12:34:20.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:20.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673062 2026-02-17T12:34:20.301 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673062 2026-02-17T12:34:20.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673062' 2026-02-17T12:34:20.302 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:20.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673062 -lt 42949673062 2026-02-17T12:34:20.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:20.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542238 2026-02-17T12:34:20.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:20.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:20.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542238 2026-02-17T12:34:20.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:20.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542238 2026-02-17T12:34:20.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542238' 2026-02-17T12:34:20.472 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542238 2026-02-17T12:34:20.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:20.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542239 -lt 60129542238 2026-02-17T12:34:20.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:20.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:20.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:20.773 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:20.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:20.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:20.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:20.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:20.784 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:20.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:20.937 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:20.937 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:20.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:20.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:20.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:21.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836592 2026-02-17T12:34:21.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836592 2026-02-17T12:34:21.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836592' 2026-02-17T12:34:21.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:21.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:21.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673065 2026-02-17T12:34:21.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673065 2026-02-17T12:34:21.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836592 1-42949673065' 2026-02-17T12:34:21.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:21.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:21.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542241 2026-02-17T12:34:21.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542241 2026-02-17T12:34:21.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836592 1-42949673065 2-60129542241' 2026-02-17T12:34:21.152 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:21.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836592 2026-02-17T12:34:21.152 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:21.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:21.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836592 2026-02-17T12:34:21.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:21.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836592 2026-02-17T12:34:21.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836592' 2026-02-17T12:34:21.156 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836592 2026-02-17T12:34:21.156 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:21.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836590 -lt 21474836592 2026-02-17T12:34:21.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:22.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:22.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:22.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836593 -lt 21474836592 2026-02-17T12:34:22.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:22.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673065 2026-02-17T12:34:22.459 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:22.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:22.461 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673065 2026-02-17T12:34:22.461 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:22.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673065 2026-02-17T12:34:22.463 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673065 2026-02-17T12:34:22.463 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673065' 2026-02-17T12:34:22.463 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:22.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673065 -lt 42949673065 2026-02-17T12:34:22.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:22.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542241 2026-02-17T12:34:22.616 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:22.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:22.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542241 2026-02-17T12:34:22.618 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:22.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542241 2026-02-17T12:34:22.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542241' 2026-02-17T12:34:22.620 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542241 2026-02-17T12:34:22.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:22.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542242 -lt 60129542241 2026-02-17T12:34:22.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:22.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:22.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:22.926 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:22.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:22.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:22.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:22.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:22.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:23.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:23.082 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:23.083 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:23.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:23.083 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:23.083 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:23.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836595 2026-02-17T12:34:23.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836595 2026-02-17T12:34:23.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836595' 2026-02-17T12:34:23.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:23.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:23.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673068 2026-02-17T12:34:23.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673068 2026-02-17T12:34:23.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836595 1-42949673068' 2026-02-17T12:34:23.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:23.227 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:23.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542244 2026-02-17T12:34:23.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542244 2026-02-17T12:34:23.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836595 1-42949673068 2-60129542244' 2026-02-17T12:34:23.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:23.300 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836595 2026-02-17T12:34:23.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:23.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:23.302 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836595 2026-02-17T12:34:23.302 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:23.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836595 2026-02-17T12:34:23.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836595' 2026-02-17T12:34:23.304 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836595 2026-02-17T12:34:23.305 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:23.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836593 -lt 21474836595 2026-02-17T12:34:23.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:24.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:24.463 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:24.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836596 -lt 21474836595 2026-02-17T12:34:24.618 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:24.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673068 2026-02-17T12:34:24.619 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:24.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:24.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673068 2026-02-17T12:34:24.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:24.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673068 2026-02-17T12:34:24.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673068' 2026-02-17T12:34:24.623 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673068 2026-02-17T12:34:24.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:24.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673068 -lt 42949673068 2026-02-17T12:34:24.773 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:24.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542244 2026-02-17T12:34:24.774 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:24.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:24.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542244 2026-02-17T12:34:24.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:24.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542244 2026-02-17T12:34:24.777 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542244 2026-02-17T12:34:24.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542244' 2026-02-17T12:34:24.777 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:24.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542245 -lt 60129542244 2026-02-17T12:34:24.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:24.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:24.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:25.079 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:25.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:25.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:25.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:25.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:25.089 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:25.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:25.238 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:25.239 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:25.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:25.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:25.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:25.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836599 2026-02-17T12:34:25.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836599 2026-02-17T12:34:25.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836599' 2026-02-17T12:34:25.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:25.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:25.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673071 2026-02-17T12:34:25.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673071 2026-02-17T12:34:25.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836599 1-42949673071' 2026-02-17T12:34:25.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:25.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:25.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542248 2026-02-17T12:34:25.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542248 2026-02-17T12:34:25.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836599 1-42949673071 2-60129542248' 2026-02-17T12:34:25.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:25.456 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836599 2026-02-17T12:34:25.456 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:25.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:25.458 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836599 2026-02-17T12:34:25.458 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:25.459 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836599 2026-02-17T12:34:25.459 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836599 2026-02-17T12:34:25.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836599' 2026-02-17T12:34:25.460 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:25.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836596 -lt 21474836599 2026-02-17T12:34:25.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:26.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:26.610 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:26.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836599 -lt 21474836599 2026-02-17T12:34:26.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:26.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673071 2026-02-17T12:34:26.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:26.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:26.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673071 2026-02-17T12:34:26.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:26.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673071 2026-02-17T12:34:26.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673071' 2026-02-17T12:34:26.773 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673071 2026-02-17T12:34:26.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:26.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673071 -lt 42949673071 2026-02-17T12:34:26.928 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:26.929 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542248 2026-02-17T12:34:26.929 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:26.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:26.931 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542248 2026-02-17T12:34:26.931 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:26.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542248 2026-02-17T12:34:26.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542248' 2026-02-17T12:34:26.933 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542248 2026-02-17T12:34:26.933 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:27.097 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542248 -lt 60129542248 2026-02-17T12:34:27.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:27.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:27.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:27.236 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:27.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:27.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:27.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:27.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:27.247 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:27.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:27.394 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:27.394 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:27.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:27.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:27.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:27.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836602 2026-02-17T12:34:27.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836602 2026-02-17T12:34:27.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836602' 2026-02-17T12:34:27.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:27.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:27.534 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673074 2026-02-17T12:34:27.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673074 2026-02-17T12:34:27.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836602 1-42949673074' 2026-02-17T12:34:27.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:27.535 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:27.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542251 2026-02-17T12:34:27.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542251 2026-02-17T12:34:27.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836602 1-42949673074 2-60129542251' 2026-02-17T12:34:27.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:27.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836602 2026-02-17T12:34:27.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:27.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:27.610 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836602 2026-02-17T12:34:27.610 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:27.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836602 2026-02-17T12:34:27.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836602' 2026-02-17T12:34:27.612 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836602 2026-02-17T12:34:27.613 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:27.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836602 -lt 21474836602 2026-02-17T12:34:27.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:27.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673074 2026-02-17T12:34:27.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:27.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:27.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673074 2026-02-17T12:34:27.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:27.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673074 2026-02-17T12:34:27.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673074' 2026-02-17T12:34:27.775 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673074 2026-02-17T12:34:27.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:27.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673074 -lt 42949673074 2026-02-17T12:34:27.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:27.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542251 2026-02-17T12:34:27.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:27.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:27.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542251 2026-02-17T12:34:27.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:27.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542251 2026-02-17T12:34:27.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542251' 2026-02-17T12:34:27.945 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542251 2026-02-17T12:34:27.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:28.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542251 -lt 60129542251 2026-02-17T12:34:28.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:28.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:28.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:28.250 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:28.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:28.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:28.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:28.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:28.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:28.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:28.406 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:28.407 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:28.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:28.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:28.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:28.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836604 2026-02-17T12:34:28.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836604 2026-02-17T12:34:28.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836604' 2026-02-17T12:34:28.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:28.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:28.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673076 2026-02-17T12:34:28.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673076 2026-02-17T12:34:28.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836604 1-42949673076' 2026-02-17T12:34:28.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:28.551 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:28.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542253 2026-02-17T12:34:28.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542253 2026-02-17T12:34:28.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836604 1-42949673076 2-60129542253' 2026-02-17T12:34:28.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:28.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836604 2026-02-17T12:34:28.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:28.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:28.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836604 2026-02-17T12:34:28.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:28.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836604 2026-02-17T12:34:28.624 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836604 2026-02-17T12:34:28.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836604' 2026-02-17T12:34:28.625 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:28.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836602 -lt 21474836604 2026-02-17T12:34:28.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:29.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:29.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:29.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836605 -lt 21474836604 2026-02-17T12:34:29.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:29.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673076 2026-02-17T12:34:29.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:29.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:29.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673076 2026-02-17T12:34:29.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:29.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673076 2026-02-17T12:34:29.945 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673076 2026-02-17T12:34:29.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673076' 2026-02-17T12:34:29.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:30.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673077 -lt 42949673076 2026-02-17T12:34:30.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:30.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542253 2026-02-17T12:34:30.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:30.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:30.110 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542253 2026-02-17T12:34:30.110 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:30.111 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542253 2026-02-17T12:34:30.111 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542253 2026-02-17T12:34:30.112 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542253' 2026-02-17T12:34:30.112 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:30.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542254 -lt 60129542253 2026-02-17T12:34:30.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:30.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:30.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:30.404 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:30.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:30.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:30.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:30.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:30.415 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:30.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:30.570 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:30.571 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:30.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:30.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:30.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:30.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836607 2026-02-17T12:34:30.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836607 2026-02-17T12:34:30.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836607' 2026-02-17T12:34:30.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:30.641 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:30.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673079 2026-02-17T12:34:30.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673079 2026-02-17T12:34:30.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836607 1-42949673079' 2026-02-17T12:34:30.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:30.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:30.782 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542256 2026-02-17T12:34:30.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542256 2026-02-17T12:34:30.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836607 1-42949673079 2-60129542256' 2026-02-17T12:34:30.783 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:30.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836607 2026-02-17T12:34:30.783 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:30.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:30.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836607 2026-02-17T12:34:30.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:30.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836607 2026-02-17T12:34:30.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836607' 2026-02-17T12:34:30.787 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836607 2026-02-17T12:34:30.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:30.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836605 -lt 21474836607 2026-02-17T12:34:30.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:31.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:31.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:32.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836608 -lt 21474836607 2026-02-17T12:34:32.094 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:32.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673079 2026-02-17T12:34:32.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:32.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:32.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673079 2026-02-17T12:34:32.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:32.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673079 2026-02-17T12:34:32.098 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673079 2026-02-17T12:34:32.098 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673079' 2026-02-17T12:34:32.098 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:32.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673080 -lt 42949673079 2026-02-17T12:34:32.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:32.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542256 2026-02-17T12:34:32.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:32.260 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:32.260 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542256 2026-02-17T12:34:32.261 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:32.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542256 2026-02-17T12:34:32.262 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542256 2026-02-17T12:34:32.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542256' 2026-02-17T12:34:32.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:32.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542257 -lt 60129542256 2026-02-17T12:34:32.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:32.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:32.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:32.561 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:32.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:32.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:32.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:32.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:32.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:32.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:32.717 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:32.717 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:32.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:32.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:32.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:32.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836610 2026-02-17T12:34:32.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836610 2026-02-17T12:34:32.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836610' 2026-02-17T12:34:32.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:32.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:32.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673083 2026-02-17T12:34:32.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673083 2026-02-17T12:34:32.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836610 1-42949673083' 2026-02-17T12:34:32.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:32.860 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:32.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542259 2026-02-17T12:34:32.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542259 2026-02-17T12:34:32.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836610 1-42949673083 2-60129542259' 2026-02-17T12:34:32.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:32.933 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836610 2026-02-17T12:34:32.933 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:32.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:32.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836610 2026-02-17T12:34:32.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:32.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836610 2026-02-17T12:34:32.936 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836610 2026-02-17T12:34:32.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836610' 2026-02-17T12:34:32.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:33.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836608 -lt 21474836610 2026-02-17T12:34:33.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:34.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:34.089 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:34.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836611 -lt 21474836610 2026-02-17T12:34:34.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:34.245 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673083 2026-02-17T12:34:34.245 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:34.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:34.247 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673083 2026-02-17T12:34:34.247 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:34.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673083 2026-02-17T12:34:34.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673083' 2026-02-17T12:34:34.249 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673083 2026-02-17T12:34:34.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:34.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673083 -lt 42949673083 2026-02-17T12:34:34.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:34.404 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542259 2026-02-17T12:34:34.404 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:34.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:34.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542259 2026-02-17T12:34:34.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:34.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542259 2026-02-17T12:34:34.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542259' 2026-02-17T12:34:34.408 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542259 2026-02-17T12:34:34.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:34.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542260 -lt 60129542259 2026-02-17T12:34:34.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:34.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:34.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:34.707 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:34.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:34.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:34.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:34.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:34.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:34.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:34.873 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:34.873 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:34.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:34.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:34.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:34.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836613 2026-02-17T12:34:34.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836613 2026-02-17T12:34:34.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836613' 2026-02-17T12:34:34.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:34.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:35.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673086 2026-02-17T12:34:35.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673086 2026-02-17T12:34:35.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836613 1-42949673086' 2026-02-17T12:34:35.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:35.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:35.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542262 2026-02-17T12:34:35.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542262 2026-02-17T12:34:35.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836613 1-42949673086 2-60129542262' 2026-02-17T12:34:35.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:35.089 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836613 2026-02-17T12:34:35.089 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:35.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:35.091 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836613 2026-02-17T12:34:35.091 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:35.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836613 2026-02-17T12:34:35.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836613' 2026-02-17T12:34:35.093 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836613 2026-02-17T12:34:35.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:35.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836611 -lt 21474836613 2026-02-17T12:34:35.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:36.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:36.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:36.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836614 -lt 21474836613 2026-02-17T12:34:36.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:36.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673086 2026-02-17T12:34:36.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:36.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:36.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673086 2026-02-17T12:34:36.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:36.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673086 2026-02-17T12:34:36.409 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673086 2026-02-17T12:34:36.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673086' 2026-02-17T12:34:36.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:36.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673086 -lt 42949673086 2026-02-17T12:34:36.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:36.564 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542262 2026-02-17T12:34:36.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:36.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:36.566 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542262 2026-02-17T12:34:36.566 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:36.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542262 2026-02-17T12:34:36.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542262' 2026-02-17T12:34:36.568 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542262 2026-02-17T12:34:36.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:36.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542263 -lt 60129542262 2026-02-17T12:34:36.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:36.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:36.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:36.871 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:36.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:36.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:36.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:36.881 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:36.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:37.035 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:37.035 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:37.036 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:37.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:37.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:37.036 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:37.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836616 2026-02-17T12:34:37.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836616 2026-02-17T12:34:37.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836616' 2026-02-17T12:34:37.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:37.106 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:37.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673089 2026-02-17T12:34:37.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673089 2026-02-17T12:34:37.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836616 1-42949673089' 2026-02-17T12:34:37.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:37.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:37.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542265 2026-02-17T12:34:37.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542265 2026-02-17T12:34:37.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836616 1-42949673089 2-60129542265' 2026-02-17T12:34:37.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:37.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836616 2026-02-17T12:34:37.252 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:37.253 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:37.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836616 2026-02-17T12:34:37.254 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:37.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836616 2026-02-17T12:34:37.256 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836616 2026-02-17T12:34:37.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836616' 2026-02-17T12:34:37.257 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:37.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836614 -lt 21474836616 2026-02-17T12:34:37.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:38.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:38.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:38.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836617 -lt 21474836616 2026-02-17T12:34:38.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:38.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673089 2026-02-17T12:34:38.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:38.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:38.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673089 2026-02-17T12:34:38.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:38.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673089 2026-02-17T12:34:38.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673089' 2026-02-17T12:34:38.567 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673089 2026-02-17T12:34:38.567 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:38.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673089 -lt 42949673089 2026-02-17T12:34:38.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:38.722 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542265 2026-02-17T12:34:38.722 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:38.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:38.723 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542265 2026-02-17T12:34:38.723 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:38.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542265 2026-02-17T12:34:38.725 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542265 2026-02-17T12:34:38.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542265' 2026-02-17T12:34:38.726 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:38.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542266 -lt 60129542265 2026-02-17T12:34:38.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:38.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:38.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:39.030 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:39.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:39.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:39.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:39.041 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:39.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:39.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:39.192 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:39.192 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:39.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:39.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:39.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:39.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836620 2026-02-17T12:34:39.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836620 2026-02-17T12:34:39.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836620' 2026-02-17T12:34:39.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:39.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:39.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673092 2026-02-17T12:34:39.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673092 2026-02-17T12:34:39.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836620 1-42949673092' 2026-02-17T12:34:39.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:39.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:39.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542269 2026-02-17T12:34:39.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542269 2026-02-17T12:34:39.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836620 1-42949673092 2-60129542269' 2026-02-17T12:34:39.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:39.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836620 2026-02-17T12:34:39.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:39.409 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:39.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836620 2026-02-17T12:34:39.410 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:39.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836620 2026-02-17T12:34:39.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836620' 2026-02-17T12:34:39.412 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836620 2026-02-17T12:34:39.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:39.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836617 -lt 21474836620 2026-02-17T12:34:39.572 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:40.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:40.574 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:40.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836620 -lt 21474836620 2026-02-17T12:34:40.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:40.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673092 2026-02-17T12:34:40.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:40.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:40.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673092 2026-02-17T12:34:40.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:40.732 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673092 2026-02-17T12:34:40.732 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673092 2026-02-17T12:34:40.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673092' 2026-02-17T12:34:40.733 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:40.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673092 -lt 42949673092 2026-02-17T12:34:40.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:40.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542269 2026-02-17T12:34:40.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:40.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:40.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542269 2026-02-17T12:34:40.902 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:40.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542269 2026-02-17T12:34:40.903 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542269 2026-02-17T12:34:40.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542269' 2026-02-17T12:34:40.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:41.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542269 -lt 60129542269 2026-02-17T12:34:41.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:41.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:41.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:41.205 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:41.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:41.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:41.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:41.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:41.215 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:41.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:41.369 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:41.370 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:41.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:41.370 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:41.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:41.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836623 2026-02-17T12:34:41.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836623 2026-02-17T12:34:41.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836623' 2026-02-17T12:34:41.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:41.441 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673095 2026-02-17T12:34:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673095 2026-02-17T12:34:41.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836623 1-42949673095' 2026-02-17T12:34:41.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:41.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:41.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542272 2026-02-17T12:34:41.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542272 2026-02-17T12:34:41.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836623 1-42949673095 2-60129542272' 2026-02-17T12:34:41.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:41.583 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836623 2026-02-17T12:34:41.584 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:41.584 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:41.585 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836623 2026-02-17T12:34:41.585 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:41.586 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836623 2026-02-17T12:34:41.587 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836623 2026-02-17T12:34:41.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836623' 2026-02-17T12:34:41.587 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:41.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836623 -lt 21474836623 2026-02-17T12:34:41.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:41.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673095 2026-02-17T12:34:41.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:41.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:41.745 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673095 2026-02-17T12:34:41.746 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:41.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673095 2026-02-17T12:34:41.747 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673095 2026-02-17T12:34:41.748 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673095' 2026-02-17T12:34:41.748 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:41.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673095 -lt 42949673095 2026-02-17T12:34:41.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:41.897 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542272 2026-02-17T12:34:41.897 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:41.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:41.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542272 2026-02-17T12:34:41.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:41.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542272 2026-02-17T12:34:41.900 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542272 2026-02-17T12:34:41.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542272' 2026-02-17T12:34:41.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:42.060 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542272 -lt 60129542272 2026-02-17T12:34:42.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:42.061 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:42.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:42.203 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:42.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:42.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:42.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:42.213 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:42.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:42.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:42.367 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:42.367 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:42.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:42.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:42.367 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:42.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836625 2026-02-17T12:34:42.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836625 2026-02-17T12:34:42.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836625' 2026-02-17T12:34:42.438 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:42.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:42.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673097 2026-02-17T12:34:42.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673097 2026-02-17T12:34:42.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836625 1-42949673097' 2026-02-17T12:34:42.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:42.509 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:42.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542274 2026-02-17T12:34:42.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542274 2026-02-17T12:34:42.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836625 1-42949673097 2-60129542274' 2026-02-17T12:34:42.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:42.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836625 2026-02-17T12:34:42.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:42.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:42.582 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836625 2026-02-17T12:34:42.582 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:42.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836625 2026-02-17T12:34:42.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836625' 2026-02-17T12:34:42.584 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836625 2026-02-17T12:34:42.584 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:42.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836623 -lt 21474836625 2026-02-17T12:34:42.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:43.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:43.747 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:43.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836626 -lt 21474836625 2026-02-17T12:34:43.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:43.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673097 2026-02-17T12:34:43.904 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:43.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:43.906 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673097 2026-02-17T12:34:43.906 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:43.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673097 2026-02-17T12:34:43.907 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673097' 2026-02-17T12:34:43.908 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673097 2026-02-17T12:34:43.908 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:44.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673098 -lt 42949673097 2026-02-17T12:34:44.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:44.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542274 2026-02-17T12:34:44.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:44.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:44.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542274 2026-02-17T12:34:44.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:44.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542274 2026-02-17T12:34:44.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542274' 2026-02-17T12:34:44.071 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542274 2026-02-17T12:34:44.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:44.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542275 -lt 60129542274 2026-02-17T12:34:44.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:44.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:44.226 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:44.362 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:44.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:44.371 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:44.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:44.372 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:44.372 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:44.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:44.521 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:44.521 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:44.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:44.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:44.521 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:44.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836628 2026-02-17T12:34:44.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836628 2026-02-17T12:34:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836628' 2026-02-17T12:34:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:44.592 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:44.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673100 2026-02-17T12:34:44.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673100 2026-02-17T12:34:44.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836628 1-42949673100' 2026-02-17T12:34:44.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:44.664 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:44.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542277 2026-02-17T12:34:44.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542277 2026-02-17T12:34:44.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836628 1-42949673100 2-60129542277' 2026-02-17T12:34:44.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:44.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836628 2026-02-17T12:34:44.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:44.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:44.741 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836628 2026-02-17T12:34:44.741 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:44.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836628 2026-02-17T12:34:44.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836628' 2026-02-17T12:34:44.743 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836628 2026-02-17T12:34:44.743 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:44.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836626 -lt 21474836628 2026-02-17T12:34:44.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:45.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:45.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:46.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836629 -lt 21474836628 2026-02-17T12:34:46.049 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:46.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673100 2026-02-17T12:34:46.050 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:46.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:46.052 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673100 2026-02-17T12:34:46.052 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:46.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673100 2026-02-17T12:34:46.054 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673100 2026-02-17T12:34:46.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673100' 2026-02-17T12:34:46.054 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:46.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673101 -lt 42949673100 2026-02-17T12:34:46.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:46.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542277 2026-02-17T12:34:46.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:46.214 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:46.215 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542277 2026-02-17T12:34:46.215 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:46.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542277 2026-02-17T12:34:46.216 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542277 2026-02-17T12:34:46.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542277' 2026-02-17T12:34:46.217 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:46.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542278 -lt 60129542277 2026-02-17T12:34:46.378 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:46.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:46.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:46.522 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:46.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:46.531 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:46.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:46.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:46.532 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:46.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:46.682 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:46.682 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:46.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:46.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:46.682 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:46.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836631 2026-02-17T12:34:46.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836631 2026-02-17T12:34:46.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836631' 2026-02-17T12:34:46.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:46.755 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:46.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673104 2026-02-17T12:34:46.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673104 2026-02-17T12:34:46.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836631 1-42949673104' 2026-02-17T12:34:46.830 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:46.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:46.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542280 2026-02-17T12:34:46.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542280 2026-02-17T12:34:46.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836631 1-42949673104 2-60129542280' 2026-02-17T12:34:46.899 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:46.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836631 2026-02-17T12:34:46.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:46.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:46.902 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836631 2026-02-17T12:34:46.902 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:46.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836631 2026-02-17T12:34:46.903 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836631 2026-02-17T12:34:46.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836631' 2026-02-17T12:34:46.910 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:47.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836629 -lt 21474836631 2026-02-17T12:34:47.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:48.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:48.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:48.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836632 -lt 21474836631 2026-02-17T12:34:48.216 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:48.217 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673104 2026-02-17T12:34:48.217 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:48.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:48.219 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673104 2026-02-17T12:34:48.219 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:48.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673104 2026-02-17T12:34:48.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673104' 2026-02-17T12:34:48.221 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673104 2026-02-17T12:34:48.221 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:48.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673104 -lt 42949673104 2026-02-17T12:34:48.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:48.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542280 2026-02-17T12:34:48.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:48.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:48.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542280 2026-02-17T12:34:48.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:48.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542280 2026-02-17T12:34:48.386 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542280 2026-02-17T12:34:48.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542280' 2026-02-17T12:34:48.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:48.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542281 -lt 60129542280 2026-02-17T12:34:48.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:48.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:48.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:48.688 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:48.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:48.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:48.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:48.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:48.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:48.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:48.849 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:48.849 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:48.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:48.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:48.850 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:48.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836634 2026-02-17T12:34:48.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836634 2026-02-17T12:34:48.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836634' 2026-02-17T12:34:48.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:48.920 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:48.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673107 2026-02-17T12:34:48.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673107 2026-02-17T12:34:48.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836634 1-42949673107' 2026-02-17T12:34:48.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:48.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:49.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542283 2026-02-17T12:34:49.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542283 2026-02-17T12:34:49.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836634 1-42949673107 2-60129542283' 2026-02-17T12:34:49.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:49.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836634 2026-02-17T12:34:49.064 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:49.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:49.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836634 2026-02-17T12:34:49.066 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:49.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836634 2026-02-17T12:34:49.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836634' 2026-02-17T12:34:49.067 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836634 2026-02-17T12:34:49.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:49.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836632 -lt 21474836634 2026-02-17T12:34:49.224 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:50.225 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:50.226 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:50.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836635 -lt 21474836634 2026-02-17T12:34:50.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:50.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673107 2026-02-17T12:34:50.381 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:50.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:50.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673107 2026-02-17T12:34:50.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:50.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673107 2026-02-17T12:34:50.384 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673107 2026-02-17T12:34:50.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673107' 2026-02-17T12:34:50.384 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:50.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673107 -lt 42949673107 2026-02-17T12:34:50.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:50.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542283 2026-02-17T12:34:50.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:50.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:50.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542283 2026-02-17T12:34:50.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:50.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542283 2026-02-17T12:34:50.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542283' 2026-02-17T12:34:50.544 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542283 2026-02-17T12:34:50.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:50.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542284 -lt 60129542283 2026-02-17T12:34:50.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:50.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:50.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:50.851 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:50.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:50.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:50.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:50.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:50.861 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:51.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:51.017 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:51.017 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:51.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:51.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:51.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:51.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836637 2026-02-17T12:34:51.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836637 2026-02-17T12:34:51.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836637' 2026-02-17T12:34:51.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:51.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:51.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673110 2026-02-17T12:34:51.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673110 2026-02-17T12:34:51.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836637 1-42949673110' 2026-02-17T12:34:51.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:51.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:51.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542286 2026-02-17T12:34:51.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542286 2026-02-17T12:34:51.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836637 1-42949673110 2-60129542286' 2026-02-17T12:34:51.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:51.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836637 2026-02-17T12:34:51.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:51.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:51.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836637 2026-02-17T12:34:51.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:51.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836637 2026-02-17T12:34:51.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836637' 2026-02-17T12:34:51.234 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836637 2026-02-17T12:34:51.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:51.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836635 -lt 21474836637 2026-02-17T12:34:51.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:52.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:52.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:52.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836638 -lt 21474836637 2026-02-17T12:34:52.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:52.549 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673110 2026-02-17T12:34:52.549 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:52.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:52.551 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673110 2026-02-17T12:34:52.551 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:52.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673110 2026-02-17T12:34:52.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673110' 2026-02-17T12:34:52.553 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673110 2026-02-17T12:34:52.553 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:52.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673110 -lt 42949673110 2026-02-17T12:34:52.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:52.716 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542286 2026-02-17T12:34:52.716 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:52.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:52.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542286 2026-02-17T12:34:52.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:52.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542286 2026-02-17T12:34:52.719 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542286 2026-02-17T12:34:52.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542286' 2026-02-17T12:34:52.720 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:52.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542287 -lt 60129542286 2026-02-17T12:34:52.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:52.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:52.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:53.010 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:53.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:53.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:53.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:53.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:53.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:53.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:53.176 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:53.176 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:53.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:53.176 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:53.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:53.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836641 2026-02-17T12:34:53.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836641 2026-02-17T12:34:53.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836641' 2026-02-17T12:34:53.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:53.247 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:53.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673113 2026-02-17T12:34:53.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673113 2026-02-17T12:34:53.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836641 1-42949673113' 2026-02-17T12:34:53.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:53.319 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:53.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542290 2026-02-17T12:34:53.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542290 2026-02-17T12:34:53.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836641 1-42949673113 2-60129542290' 2026-02-17T12:34:53.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:53.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836641 2026-02-17T12:34:53.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:53.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:53.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836641 2026-02-17T12:34:53.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:53.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836641 2026-02-17T12:34:53.391 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836641 2026-02-17T12:34:53.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836641' 2026-02-17T12:34:53.392 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:53.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836638 -lt 21474836641 2026-02-17T12:34:53.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:54.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:54.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:54.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836641 -lt 21474836641 2026-02-17T12:34:54.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:54.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673113 2026-02-17T12:34:54.698 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:54.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:54.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673113 2026-02-17T12:34:54.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:54.701 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673113 2026-02-17T12:34:54.702 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673113 2026-02-17T12:34:54.702 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673113' 2026-02-17T12:34:54.702 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:54.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673113 -lt 42949673113 2026-02-17T12:34:54.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:54.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542290 2026-02-17T12:34:54.869 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:54.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:54.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542290 2026-02-17T12:34:54.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:54.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542290 2026-02-17T12:34:54.872 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542290' 2026-02-17T12:34:54.872 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542290 2026-02-17T12:34:54.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:55.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542290 -lt 60129542290 2026-02-17T12:34:55.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:55.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:55.031 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:55.175 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:55.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:55.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:55.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:55.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:55.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:55.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:55.338 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:55.339 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:55.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:55.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:55.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:55.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836644 2026-02-17T12:34:55.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836644 2026-02-17T12:34:55.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836644' 2026-02-17T12:34:55.408 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:55.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:55.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673116 2026-02-17T12:34:55.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673116 2026-02-17T12:34:55.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836644 1-42949673116' 2026-02-17T12:34:55.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:55.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:55.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542293 2026-02-17T12:34:55.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542293 2026-02-17T12:34:55.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836644 1-42949673116 2-60129542293' 2026-02-17T12:34:55.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:55.552 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836644 2026-02-17T12:34:55.552 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:55.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:55.553 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836644 2026-02-17T12:34:55.553 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:55.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836644 2026-02-17T12:34:55.555 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836644 2026-02-17T12:34:55.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836644' 2026-02-17T12:34:55.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:55.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836644 -lt 21474836644 2026-02-17T12:34:55.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:55.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673116 2026-02-17T12:34:55.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:55.714 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:55.714 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673116 2026-02-17T12:34:55.714 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:55.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673116 2026-02-17T12:34:55.716 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673116 2026-02-17T12:34:55.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673116' 2026-02-17T12:34:55.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:55.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673116 -lt 42949673116 2026-02-17T12:34:55.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:55.875 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542293 2026-02-17T12:34:55.875 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:55.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:55.877 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542293 2026-02-17T12:34:55.877 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:55.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542293 2026-02-17T12:34:55.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542293' 2026-02-17T12:34:55.879 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542293 2026-02-17T12:34:55.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:56.026 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542293 -lt 60129542293 2026-02-17T12:34:56.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:56.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:56.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:56.171 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:56.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:56.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:56.180 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:56.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:56.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:56.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:56.333 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:56.334 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:56.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:56.334 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:56.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:56.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836646 2026-02-17T12:34:56.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836646 2026-02-17T12:34:56.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836646' 2026-02-17T12:34:56.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:56.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:56.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673118 2026-02-17T12:34:56.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673118 2026-02-17T12:34:56.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836646 1-42949673118' 2026-02-17T12:34:56.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:56.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:56.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542295 2026-02-17T12:34:56.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542295 2026-02-17T12:34:56.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836646 1-42949673118 2-60129542295' 2026-02-17T12:34:56.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:56.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836646 2026-02-17T12:34:56.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:56.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:56.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836646 2026-02-17T12:34:56.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:56.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836646 2026-02-17T12:34:56.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836646' 2026-02-17T12:34:56.552 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836646 2026-02-17T12:34:56.552 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:56.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836644 -lt 21474836646 2026-02-17T12:34:56.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:57.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:57.705 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:57.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836647 -lt 21474836646 2026-02-17T12:34:57.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:57.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673118 2026-02-17T12:34:57.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:57.860 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:34:57.861 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673118 2026-02-17T12:34:57.861 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:57.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673118 2026-02-17T12:34:57.863 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673118 2026-02-17T12:34:57.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673118' 2026-02-17T12:34:57.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:34:58.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673119 -lt 42949673118 2026-02-17T12:34:58.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:58.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542295 2026-02-17T12:34:58.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:58.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:34:58.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542295 2026-02-17T12:34:58.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:58.018 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542295 2026-02-17T12:34:58.019 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542295 2026-02-17T12:34:58.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542295' 2026-02-17T12:34:58.019 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:34:58.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542296 -lt 60129542295 2026-02-17T12:34:58.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:34:58.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:34:58.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:34:58.314 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:34:58.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:34:58.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:34:58.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:34:58.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:34:58.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:34:58.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:34:58.470 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:34:58.470 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:34:58.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:34:58.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:58.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:34:58.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836649 2026-02-17T12:34:58.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836649 2026-02-17T12:34:58.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836649' 2026-02-17T12:34:58.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:58.541 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:34:58.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673121 2026-02-17T12:34:58.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673121 2026-02-17T12:34:58.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836649 1-42949673121' 2026-02-17T12:34:58.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:34:58.610 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:34:58.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542298 2026-02-17T12:34:58.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542298 2026-02-17T12:34:58.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836649 1-42949673121 2-60129542298' 2026-02-17T12:34:58.681 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:34:58.681 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836649 2026-02-17T12:34:58.681 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:34:58.682 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:34:58.683 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836649 2026-02-17T12:34:58.683 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:34:58.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836649 2026-02-17T12:34:58.685 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836649 2026-02-17T12:34:58.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836649' 2026-02-17T12:34:58.686 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:34:58.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836647 -lt 21474836649 2026-02-17T12:34:58.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:34:59.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:34:59.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:00.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836650 -lt 21474836649 2026-02-17T12:35:00.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:00.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673121 2026-02-17T12:35:00.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:00.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:00.003 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673121 2026-02-17T12:35:00.003 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:00.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673121 2026-02-17T12:35:00.005 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673121 2026-02-17T12:35:00.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673121' 2026-02-17T12:35:00.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:00.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673122 -lt 42949673121 2026-02-17T12:35:00.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:00.164 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542298 2026-02-17T12:35:00.164 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:00.165 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:00.166 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542298 2026-02-17T12:35:00.166 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:00.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542298 2026-02-17T12:35:00.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542298' 2026-02-17T12:35:00.168 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542298 2026-02-17T12:35:00.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:00.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542299 -lt 60129542298 2026-02-17T12:35:00.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:00.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:00.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:00.469 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:00.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:00.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:00.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:00.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:00.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:00.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:00.628 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:00.629 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:00.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:00.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:00.629 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:00.699 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836652 2026-02-17T12:35:00.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836652 2026-02-17T12:35:00.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836652' 2026-02-17T12:35:00.700 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:00.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:00.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673125 2026-02-17T12:35:00.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673125 2026-02-17T12:35:00.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836652 1-42949673125' 2026-02-17T12:35:00.771 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:00.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:00.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542301 2026-02-17T12:35:00.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542301 2026-02-17T12:35:00.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836652 1-42949673125 2-60129542301' 2026-02-17T12:35:00.844 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:00.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836652 2026-02-17T12:35:00.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:00.845 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:00.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836652 2026-02-17T12:35:00.846 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:00.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836652 2026-02-17T12:35:00.847 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836652' 2026-02-17T12:35:00.847 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836652 2026-02-17T12:35:00.848 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:01.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836650 -lt 21474836652 2026-02-17T12:35:01.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:02.001 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:02.001 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:02.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836653 -lt 21474836652 2026-02-17T12:35:02.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:02.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673125 2026-02-17T12:35:02.157 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:02.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:02.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673125 2026-02-17T12:35:02.159 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:02.160 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673125 2026-02-17T12:35:02.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673125' 2026-02-17T12:35:02.161 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673125 2026-02-17T12:35:02.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:02.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673125 -lt 42949673125 2026-02-17T12:35:02.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:02.319 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542301 2026-02-17T12:35:02.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:02.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:02.321 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542301 2026-02-17T12:35:02.321 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:02.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542301 2026-02-17T12:35:02.323 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542301 2026-02-17T12:35:02.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542301' 2026-02-17T12:35:02.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:02.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542302 -lt 60129542301 2026-02-17T12:35:02.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:02.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:02.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:02.619 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:02.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:02.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:02.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:02.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:02.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:02.778 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:02.778 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:02.779 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:02.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:02.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:02.779 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:02.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836655 2026-02-17T12:35:02.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836655 2026-02-17T12:35:02.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836655' 2026-02-17T12:35:02.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:02.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:02.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673128 2026-02-17T12:35:02.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673128 2026-02-17T12:35:02.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836655 1-42949673128' 2026-02-17T12:35:02.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:02.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:02.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542304 2026-02-17T12:35:02.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542304 2026-02-17T12:35:02.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836655 1-42949673128 2-60129542304' 2026-02-17T12:35:02.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:02.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836655 2026-02-17T12:35:02.997 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:02.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:02.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836655 2026-02-17T12:35:02.998 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:02.999 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836655 2026-02-17T12:35:03.000 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836655 2026-02-17T12:35:03.000 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836655' 2026-02-17T12:35:03.000 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:03.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836653 -lt 21474836655 2026-02-17T12:35:03.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:04.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:04.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:04.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836656 -lt 21474836655 2026-02-17T12:35:04.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:04.315 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673128 2026-02-17T12:35:04.315 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:04.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:04.316 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673128 2026-02-17T12:35:04.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:04.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673128 2026-02-17T12:35:04.318 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673128 2026-02-17T12:35:04.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673128' 2026-02-17T12:35:04.319 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:04.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673128 -lt 42949673128 2026-02-17T12:35:04.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:04.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542304 2026-02-17T12:35:04.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:04.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:04.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542304 2026-02-17T12:35:04.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:04.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542304 2026-02-17T12:35:04.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542304' 2026-02-17T12:35:04.473 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542304 2026-02-17T12:35:04.474 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:04.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542305 -lt 60129542304 2026-02-17T12:35:04.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:04.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:04.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:04.780 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:04.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:04.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:04.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:04.789 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:04.790 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:04.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:04.937 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:04.937 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:04.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:04.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:04.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:05.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836658 2026-02-17T12:35:05.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836658 2026-02-17T12:35:05.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836658' 2026-02-17T12:35:05.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:05.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:05.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673131 2026-02-17T12:35:05.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673131 2026-02-17T12:35:05.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836658 1-42949673131' 2026-02-17T12:35:05.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:05.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:05.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542307 2026-02-17T12:35:05.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542307 2026-02-17T12:35:05.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836658 1-42949673131 2-60129542307' 2026-02-17T12:35:05.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:05.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836658 2026-02-17T12:35:05.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:05.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:05.156 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836658 2026-02-17T12:35:05.156 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:05.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836658 2026-02-17T12:35:05.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836658' 2026-02-17T12:35:05.158 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836658 2026-02-17T12:35:05.158 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:05.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836656 -lt 21474836658 2026-02-17T12:35:05.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:06.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:06.316 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:06.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836659 -lt 21474836658 2026-02-17T12:35:06.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:06.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673131 2026-02-17T12:35:06.475 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:06.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:06.476 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673131 2026-02-17T12:35:06.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:06.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673131 2026-02-17T12:35:06.478 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673131' 2026-02-17T12:35:06.479 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673131 2026-02-17T12:35:06.479 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:06.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673131 -lt 42949673131 2026-02-17T12:35:06.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:06.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542307 2026-02-17T12:35:06.629 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:06.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:06.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542307 2026-02-17T12:35:06.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:06.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542307 2026-02-17T12:35:06.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542307' 2026-02-17T12:35:06.632 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542307 2026-02-17T12:35:06.632 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:06.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542308 -lt 60129542307 2026-02-17T12:35:06.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:06.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:06.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:06.936 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:06.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:06.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:06.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:06.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:06.946 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:07.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:07.093 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:07.093 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:07.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:07.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:07.094 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:07.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836661 2026-02-17T12:35:07.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836661 2026-02-17T12:35:07.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836661' 2026-02-17T12:35:07.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:07.165 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:07.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673134 2026-02-17T12:35:07.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673134 2026-02-17T12:35:07.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836661 1-42949673134' 2026-02-17T12:35:07.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:07.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:07.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542310 2026-02-17T12:35:07.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542310 2026-02-17T12:35:07.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836661 1-42949673134 2-60129542310' 2026-02-17T12:35:07.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:07.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836661 2026-02-17T12:35:07.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:07.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:07.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836661 2026-02-17T12:35:07.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:07.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836661 2026-02-17T12:35:07.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836661' 2026-02-17T12:35:07.312 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836661 2026-02-17T12:35:07.312 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:07.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836659 -lt 21474836661 2026-02-17T12:35:07.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:08.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:08.474 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:08.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836662 -lt 21474836661 2026-02-17T12:35:08.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:08.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673134 2026-02-17T12:35:08.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:08.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:08.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673134 2026-02-17T12:35:08.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:08.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673134 2026-02-17T12:35:08.632 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673134 2026-02-17T12:35:08.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673134' 2026-02-17T12:35:08.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:08.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673134 -lt 42949673134 2026-02-17T12:35:08.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:08.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542310 2026-02-17T12:35:08.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:08.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:08.793 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542310 2026-02-17T12:35:08.793 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:08.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542310 2026-02-17T12:35:08.794 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542310 2026-02-17T12:35:08.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542310' 2026-02-17T12:35:08.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:08.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542311 -lt 60129542310 2026-02-17T12:35:08.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:08.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:08.953 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:09.092 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:09.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:09.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:09.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:09.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:09.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:09.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:09.248 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:09.248 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:09.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:09.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:09.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:09.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836665 2026-02-17T12:35:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836665 2026-02-17T12:35:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836665' 2026-02-17T12:35:09.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:09.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:09.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673137 2026-02-17T12:35:09.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673137 2026-02-17T12:35:09.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836665 1-42949673137' 2026-02-17T12:35:09.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:09.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:09.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542314 2026-02-17T12:35:09.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542314 2026-02-17T12:35:09.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836665 1-42949673137 2-60129542314' 2026-02-17T12:35:09.464 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:09.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836665 2026-02-17T12:35:09.465 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:09.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:09.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836665 2026-02-17T12:35:09.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:09.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836665 2026-02-17T12:35:09.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836665' 2026-02-17T12:35:09.469 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836665 2026-02-17T12:35:09.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:09.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836662 -lt 21474836665 2026-02-17T12:35:09.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:10.629 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:10.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:10.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836665 -lt 21474836665 2026-02-17T12:35:10.784 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:10.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673137 2026-02-17T12:35:10.785 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:10.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:10.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673137 2026-02-17T12:35:10.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:10.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673137 2026-02-17T12:35:10.788 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673137' 2026-02-17T12:35:10.788 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673137 2026-02-17T12:35:10.789 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:10.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673137 -lt 42949673137 2026-02-17T12:35:10.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:10.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542314 2026-02-17T12:35:10.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:10.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:10.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542314 2026-02-17T12:35:10.949 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:10.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542314 2026-02-17T12:35:10.950 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542314 2026-02-17T12:35:10.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542314' 2026-02-17T12:35:10.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:11.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542314 -lt 60129542314 2026-02-17T12:35:11.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:11.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:11.108 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:11.249 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:11.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:11.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:11.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:11.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:11.259 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:11.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:11.404 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:11.404 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:11.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:11.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:11.404 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:11.476 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836668 2026-02-17T12:35:11.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836668 2026-02-17T12:35:11.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836668' 2026-02-17T12:35:11.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:11.477 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:11.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673140 2026-02-17T12:35:11.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673140 2026-02-17T12:35:11.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836668 1-42949673140' 2026-02-17T12:35:11.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:11.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:11.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542317 2026-02-17T12:35:11.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542317 2026-02-17T12:35:11.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836668 1-42949673140 2-60129542317' 2026-02-17T12:35:11.619 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:11.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836668 2026-02-17T12:35:11.620 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:11.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:11.622 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836668 2026-02-17T12:35:11.622 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:11.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836668 2026-02-17T12:35:11.623 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836668 2026-02-17T12:35:11.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836668' 2026-02-17T12:35:11.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:11.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836668 -lt 21474836668 2026-02-17T12:35:11.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:11.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673140 2026-02-17T12:35:11.771 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:11.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:11.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673140 2026-02-17T12:35:11.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:11.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673140 2026-02-17T12:35:11.774 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673140' 2026-02-17T12:35:11.775 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673140 2026-02-17T12:35:11.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:11.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673140 -lt 42949673140 2026-02-17T12:35:11.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:11.933 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542317 2026-02-17T12:35:11.933 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:11.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:11.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542317 2026-02-17T12:35:11.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:11.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542317 2026-02-17T12:35:11.937 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542317 2026-02-17T12:35:11.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542317' 2026-02-17T12:35:11.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:12.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542317 -lt 60129542317 2026-02-17T12:35:12.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:12.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:12.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:12.228 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:12.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:12.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:12.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:12.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:12.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:12.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:12.383 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:12.383 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:12.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:12.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:12.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:12.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836670 2026-02-17T12:35:12.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836670 2026-02-17T12:35:12.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836670' 2026-02-17T12:35:12.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:12.456 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:12.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673142 2026-02-17T12:35:12.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673142 2026-02-17T12:35:12.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836670 1-42949673142' 2026-02-17T12:35:12.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:12.527 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:12.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542319 2026-02-17T12:35:12.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542319 2026-02-17T12:35:12.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836670 1-42949673142 2-60129542319' 2026-02-17T12:35:12.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:12.596 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836670 2026-02-17T12:35:12.596 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:12.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:12.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836670 2026-02-17T12:35:12.599 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:12.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836670 2026-02-17T12:35:12.600 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836670 2026-02-17T12:35:12.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836670' 2026-02-17T12:35:12.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:12.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836668 -lt 21474836670 2026-02-17T12:35:12.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:13.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:13.764 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:13.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836671 -lt 21474836670 2026-02-17T12:35:13.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:13.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673142 2026-02-17T12:35:13.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:13.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:13.923 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673142 2026-02-17T12:35:13.924 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:13.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673142 2026-02-17T12:35:13.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673142' 2026-02-17T12:35:13.925 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673142 2026-02-17T12:35:13.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:14.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673143 -lt 42949673142 2026-02-17T12:35:14.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:14.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542319 2026-02-17T12:35:14.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:14.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:14.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542319 2026-02-17T12:35:14.089 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:14.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542319 2026-02-17T12:35:14.090 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542319 2026-02-17T12:35:14.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542319' 2026-02-17T12:35:14.090 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:14.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542320 -lt 60129542319 2026-02-17T12:35:14.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:14.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:14.252 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:14.393 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:14.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:14.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:14.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:14.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:14.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:14.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:14.551 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:14.551 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:14.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:14.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:14.552 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:14.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836673 2026-02-17T12:35:14.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836673 2026-02-17T12:35:14.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836673' 2026-02-17T12:35:14.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:14.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:14.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673145 2026-02-17T12:35:14.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673145 2026-02-17T12:35:14.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836673 1-42949673145' 2026-02-17T12:35:14.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:14.697 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:14.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542322 2026-02-17T12:35:14.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542322 2026-02-17T12:35:14.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836673 1-42949673145 2-60129542322' 2026-02-17T12:35:14.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:14.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836673 2026-02-17T12:35:14.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:14.770 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:14.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836673 2026-02-17T12:35:14.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:14.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836673 2026-02-17T12:35:14.772 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836673' 2026-02-17T12:35:14.772 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836673 2026-02-17T12:35:14.773 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:14.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836671 -lt 21474836673 2026-02-17T12:35:14.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:15.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:15.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:16.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836674 -lt 21474836673 2026-02-17T12:35:16.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:16.085 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673145 2026-02-17T12:35:16.085 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:16.087 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:16.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673145 2026-02-17T12:35:16.088 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:16.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673145 2026-02-17T12:35:16.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673145' 2026-02-17T12:35:16.090 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673145 2026-02-17T12:35:16.090 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:16.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673146 -lt 42949673145 2026-02-17T12:35:16.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:16.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542322 2026-02-17T12:35:16.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:16.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:16.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542322 2026-02-17T12:35:16.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:16.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542322 2026-02-17T12:35:16.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542322' 2026-02-17T12:35:16.248 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542322 2026-02-17T12:35:16.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:16.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542323 -lt 60129542322 2026-02-17T12:35:16.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:16.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:16.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:16.553 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:16.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:16.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:16.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:16.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:16.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:16.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:16.709 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:16.709 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:16.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:16.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:16.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:16.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836676 2026-02-17T12:35:16.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836676 2026-02-17T12:35:16.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836676' 2026-02-17T12:35:16.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:16.782 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:16.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673149 2026-02-17T12:35:16.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673149 2026-02-17T12:35:16.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836676 1-42949673149' 2026-02-17T12:35:16.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:16.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:16.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542325 2026-02-17T12:35:16.923 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542325 2026-02-17T12:35:16.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836676 1-42949673149 2-60129542325' 2026-02-17T12:35:16.924 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:16.924 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836676 2026-02-17T12:35:16.924 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:16.925 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:16.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836676 2026-02-17T12:35:16.926 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:16.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836676 2026-02-17T12:35:16.927 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836676' 2026-02-17T12:35:16.927 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836676 2026-02-17T12:35:16.927 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:17.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836674 -lt 21474836676 2026-02-17T12:35:17.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:18.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:18.081 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:18.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836677 -lt 21474836676 2026-02-17T12:35:18.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:18.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673149 2026-02-17T12:35:18.238 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:18.239 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:18.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673149 2026-02-17T12:35:18.240 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:18.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673149 2026-02-17T12:35:18.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673149' 2026-02-17T12:35:18.241 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673149 2026-02-17T12:35:18.241 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:18.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673149 -lt 42949673149 2026-02-17T12:35:18.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:18.393 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542325 2026-02-17T12:35:18.393 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:18.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:18.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542325 2026-02-17T12:35:18.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:18.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542325 2026-02-17T12:35:18.396 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542325 2026-02-17T12:35:18.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542325' 2026-02-17T12:35:18.397 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:18.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542326 -lt 60129542325 2026-02-17T12:35:18.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:18.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:18.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:18.700 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:18.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:18.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:18.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:18.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:18.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:18.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:18.858 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:18.859 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:18.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:18.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:18.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:18.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836679 2026-02-17T12:35:18.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836679 2026-02-17T12:35:18.932 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836679' 2026-02-17T12:35:18.933 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:18.933 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:19.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673152 2026-02-17T12:35:19.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673152 2026-02-17T12:35:19.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836679 1-42949673152' 2026-02-17T12:35:19.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:19.003 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:19.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542328 2026-02-17T12:35:19.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542328 2026-02-17T12:35:19.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836679 1-42949673152 2-60129542328' 2026-02-17T12:35:19.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:19.078 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836679 2026-02-17T12:35:19.078 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:19.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:19.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836679 2026-02-17T12:35:19.080 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:19.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836679 2026-02-17T12:35:19.082 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836679' 2026-02-17T12:35:19.082 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836679 2026-02-17T12:35:19.082 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:19.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836677 -lt 21474836679 2026-02-17T12:35:19.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:20.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:20.251 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:20.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836680 -lt 21474836679 2026-02-17T12:35:20.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:20.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673152 2026-02-17T12:35:20.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:20.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:20.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673152 2026-02-17T12:35:20.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:20.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673152 2026-02-17T12:35:20.407 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673152 2026-02-17T12:35:20.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673152' 2026-02-17T12:35:20.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:20.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673152 -lt 42949673152 2026-02-17T12:35:20.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:20.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542328 2026-02-17T12:35:20.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:20.566 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:20.567 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542328 2026-02-17T12:35:20.567 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:20.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542328 2026-02-17T12:35:20.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542328' 2026-02-17T12:35:20.568 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542328 2026-02-17T12:35:20.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:20.725 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542329 -lt 60129542328 2026-02-17T12:35:20.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:20.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:20.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:20.868 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:20.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:20.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:20.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:20.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:20.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:21.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:21.030 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:21.030 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:21.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:21.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:21.030 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:21.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836682 2026-02-17T12:35:21.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836682 2026-02-17T12:35:21.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836682' 2026-02-17T12:35:21.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:21.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:21.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673155 2026-02-17T12:35:21.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673155 2026-02-17T12:35:21.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836682 1-42949673155' 2026-02-17T12:35:21.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:21.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:21.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542331 2026-02-17T12:35:21.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542331 2026-02-17T12:35:21.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836682 1-42949673155 2-60129542331' 2026-02-17T12:35:21.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:21.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836682 2026-02-17T12:35:21.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:21.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:21.245 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836682 2026-02-17T12:35:21.245 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:21.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836682 2026-02-17T12:35:21.247 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836682 2026-02-17T12:35:21.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836682' 2026-02-17T12:35:21.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:21.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836680 -lt 21474836682 2026-02-17T12:35:21.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:22.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:22.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:22.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836683 -lt 21474836682 2026-02-17T12:35:22.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:22.554 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673155 2026-02-17T12:35:22.554 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:22.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:22.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673155 2026-02-17T12:35:22.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:22.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673155 2026-02-17T12:35:22.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673155' 2026-02-17T12:35:22.558 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673155 2026-02-17T12:35:22.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:22.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673155 -lt 42949673155 2026-02-17T12:35:22.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:22.723 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542331 2026-02-17T12:35:22.723 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:22.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:22.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542331 2026-02-17T12:35:22.725 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:22.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542331 2026-02-17T12:35:22.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542331' 2026-02-17T12:35:22.727 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542331 2026-02-17T12:35:22.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:22.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542332 -lt 60129542331 2026-02-17T12:35:22.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:22.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:22.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:23.029 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:23.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:23.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:23.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:23.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:23.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:23.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:23.188 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:23.188 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:23.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:23.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:23.188 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:23.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836686 2026-02-17T12:35:23.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836686 2026-02-17T12:35:23.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836686' 2026-02-17T12:35:23.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:23.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:23.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673158 2026-02-17T12:35:23.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673158 2026-02-17T12:35:23.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836686 1-42949673158' 2026-02-17T12:35:23.329 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:23.329 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:23.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542335 2026-02-17T12:35:23.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542335 2026-02-17T12:35:23.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836686 1-42949673158 2-60129542335' 2026-02-17T12:35:23.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:23.399 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836686 2026-02-17T12:35:23.399 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:23.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:23.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836686 2026-02-17T12:35:23.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:23.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836686 2026-02-17T12:35:23.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836686' 2026-02-17T12:35:23.402 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836686 2026-02-17T12:35:23.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:23.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836683 -lt 21474836686 2026-02-17T12:35:23.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:24.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:24.553 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:24.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836686 -lt 21474836686 2026-02-17T12:35:24.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:24.714 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673158 2026-02-17T12:35:24.714 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:24.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:24.716 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673158 2026-02-17T12:35:24.716 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:24.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673158 2026-02-17T12:35:24.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673158' 2026-02-17T12:35:24.718 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673158 2026-02-17T12:35:24.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:24.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673158 -lt 42949673158 2026-02-17T12:35:24.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:24.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542335 2026-02-17T12:35:24.866 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:24.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:24.867 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542335 2026-02-17T12:35:24.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:24.869 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542335 2026-02-17T12:35:24.869 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542335 2026-02-17T12:35:24.870 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542335' 2026-02-17T12:35:24.870 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:25.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542335 -lt 60129542335 2026-02-17T12:35:25.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:25.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:25.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:25.171 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:25.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:25.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:25.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:25.181 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:25.181 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:25.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:25.328 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:25.328 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:25.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:25.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:25.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:25.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836689 2026-02-17T12:35:25.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836689 2026-02-17T12:35:25.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836689' 2026-02-17T12:35:25.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:25.399 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:25.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673161 2026-02-17T12:35:25.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673161 2026-02-17T12:35:25.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836689 1-42949673161' 2026-02-17T12:35:25.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:25.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:25.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542338 2026-02-17T12:35:25.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542338 2026-02-17T12:35:25.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836689 1-42949673161 2-60129542338' 2026-02-17T12:35:25.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:25.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836689 2026-02-17T12:35:25.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:25.545 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:25.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836689 2026-02-17T12:35:25.546 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:25.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836689 2026-02-17T12:35:25.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836689' 2026-02-17T12:35:25.548 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836689 2026-02-17T12:35:25.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:25.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836686 -lt 21474836689 2026-02-17T12:35:25.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:26.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:26.696 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:26.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836689 -lt 21474836689 2026-02-17T12:35:26.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:26.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673161 2026-02-17T12:35:26.854 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:26.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:26.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673161 2026-02-17T12:35:26.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:26.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673161 2026-02-17T12:35:26.858 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673161 2026-02-17T12:35:26.858 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673161' 2026-02-17T12:35:26.858 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:27.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673161 -lt 42949673161 2026-02-17T12:35:27.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:27.021 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542338 2026-02-17T12:35:27.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:27.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:27.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542338 2026-02-17T12:35:27.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:27.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542338 2026-02-17T12:35:27.025 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542338 2026-02-17T12:35:27.025 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542338' 2026-02-17T12:35:27.026 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:27.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542338 -lt 60129542338 2026-02-17T12:35:27.189 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:27.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:27.190 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:27.326 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:27.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:27.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:27.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:27.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:27.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:27.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:27.490 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:27.490 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:27.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:27.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:27.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:27.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836692 2026-02-17T12:35:27.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836692 2026-02-17T12:35:27.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836692' 2026-02-17T12:35:27.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:27.563 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:27.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673164 2026-02-17T12:35:27.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673164 2026-02-17T12:35:27.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836692 1-42949673164' 2026-02-17T12:35:27.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:27.632 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:27.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542341 2026-02-17T12:35:27.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542341 2026-02-17T12:35:27.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836692 1-42949673164 2-60129542341' 2026-02-17T12:35:27.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:27.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836692 2026-02-17T12:35:27.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:27.705 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:27.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836692 2026-02-17T12:35:27.706 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:27.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836692 2026-02-17T12:35:27.708 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836692' 2026-02-17T12:35:27.708 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836692 2026-02-17T12:35:27.708 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:27.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836692 -lt 21474836692 2026-02-17T12:35:27.871 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:27.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673164 2026-02-17T12:35:27.872 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:27.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:27.873 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673164 2026-02-17T12:35:27.874 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:27.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673164 2026-02-17T12:35:27.875 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673164 2026-02-17T12:35:27.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673164' 2026-02-17T12:35:27.876 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:28.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673164 -lt 42949673164 2026-02-17T12:35:28.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:28.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542341 2026-02-17T12:35:28.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:28.030 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:28.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542341 2026-02-17T12:35:28.031 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:28.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542341 2026-02-17T12:35:28.032 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542341 2026-02-17T12:35:28.033 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542341' 2026-02-17T12:35:28.033 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:28.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542340 -lt 60129542341 2026-02-17T12:35:28.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:29.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:29.197 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:29.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542340 -lt 60129542341 2026-02-17T12:35:29.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:30.348 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-17T12:35:30.349 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:30.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542343 -lt 60129542341 2026-02-17T12:35:30.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:30.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:30.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:30.647 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:30.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:30.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:30.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:30.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:30.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:30.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:30.804 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:30.804 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:30.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:30.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:30.804 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:30.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836696 2026-02-17T12:35:30.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836696 2026-02-17T12:35:30.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836696' 2026-02-17T12:35:30.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:30.876 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:30.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673169 2026-02-17T12:35:30.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673169 2026-02-17T12:35:30.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836696 1-42949673169' 2026-02-17T12:35:30.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:30.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:31.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542345 2026-02-17T12:35:31.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542345 2026-02-17T12:35:31.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836696 1-42949673169 2-60129542345' 2026-02-17T12:35:31.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:31.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836696 2026-02-17T12:35:31.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:31.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:31.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836696 2026-02-17T12:35:31.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:31.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836696 2026-02-17T12:35:31.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836696' 2026-02-17T12:35:31.024 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836696 2026-02-17T12:35:31.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:31.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836694 -lt 21474836696 2026-02-17T12:35:31.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:32.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:32.184 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:32.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836697 -lt 21474836696 2026-02-17T12:35:32.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:32.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673169 2026-02-17T12:35:32.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:32.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:32.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673169 2026-02-17T12:35:32.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:32.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673169 2026-02-17T12:35:32.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673169' 2026-02-17T12:35:32.343 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673169 2026-02-17T12:35:32.343 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:32.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673169 -lt 42949673169 2026-02-17T12:35:32.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:32.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542345 2026-02-17T12:35:32.492 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:32.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:32.494 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542345 2026-02-17T12:35:32.494 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:32.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542345 2026-02-17T12:35:32.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542345' 2026-02-17T12:35:32.496 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542345 2026-02-17T12:35:32.496 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:32.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542346 -lt 60129542345 2026-02-17T12:35:32.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:32.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:32.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:32.792 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:32.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:32.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:32.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:32.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:32.802 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:32.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:32.950 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:32.950 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:32.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:32.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:32.951 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:33.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836699 2026-02-17T12:35:33.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836699 2026-02-17T12:35:33.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836699' 2026-02-17T12:35:33.023 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:33.024 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:33.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673172 2026-02-17T12:35:33.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673172 2026-02-17T12:35:33.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836699 1-42949673172' 2026-02-17T12:35:33.096 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:33.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:33.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542348 2026-02-17T12:35:33.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542348 2026-02-17T12:35:33.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836699 1-42949673172 2-60129542348' 2026-02-17T12:35:33.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:33.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836699 2026-02-17T12:35:33.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:33.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:33.170 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836699 2026-02-17T12:35:33.170 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:33.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836699 2026-02-17T12:35:33.172 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836699 2026-02-17T12:35:33.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836699' 2026-02-17T12:35:33.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:33.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836697 -lt 21474836699 2026-02-17T12:35:33.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:34.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:34.324 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:34.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836700 -lt 21474836699 2026-02-17T12:35:34.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:34.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673172 2026-02-17T12:35:34.478 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:34.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:34.480 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673172 2026-02-17T12:35:34.480 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:34.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673172 2026-02-17T12:35:34.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673172' 2026-02-17T12:35:34.482 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673172 2026-02-17T12:35:34.482 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:34.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673172 -lt 42949673172 2026-02-17T12:35:34.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:34.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542348 2026-02-17T12:35:34.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:34.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:34.646 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542348 2026-02-17T12:35:34.646 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:34.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542348 2026-02-17T12:35:34.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542348' 2026-02-17T12:35:34.648 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542348 2026-02-17T12:35:34.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:34.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542349 -lt 60129542348 2026-02-17T12:35:34.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:34.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:34.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:34.954 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:34.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:34.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:34.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:34.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:34.966 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:35.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:35.122 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:35.122 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:35.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:35.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:35.122 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:35.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836702 2026-02-17T12:35:35.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836702 2026-02-17T12:35:35.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836702' 2026-02-17T12:35:35.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:35.192 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:35.264 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673175 2026-02-17T12:35:35.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673175 2026-02-17T12:35:35.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836702 1-42949673175' 2026-02-17T12:35:35.265 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:35.265 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:35.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542351 2026-02-17T12:35:35.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542351 2026-02-17T12:35:35.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836702 1-42949673175 2-60129542351' 2026-02-17T12:35:35.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:35.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836702 2026-02-17T12:35:35.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:35.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:35.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836702 2026-02-17T12:35:35.337 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:35.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836702 2026-02-17T12:35:35.339 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836702 2026-02-17T12:35:35.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836702' 2026-02-17T12:35:35.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:35.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836700 -lt 21474836702 2026-02-17T12:35:35.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:36.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:36.502 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:36.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836703 -lt 21474836702 2026-02-17T12:35:36.656 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:36.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673175 2026-02-17T12:35:36.656 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:36.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:36.658 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673175 2026-02-17T12:35:36.658 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:36.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673175 2026-02-17T12:35:36.659 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673175' 2026-02-17T12:35:36.660 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673175 2026-02-17T12:35:36.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:36.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673175 -lt 42949673175 2026-02-17T12:35:36.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:36.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542351 2026-02-17T12:35:36.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:36.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:36.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542351 2026-02-17T12:35:36.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:36.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542351 2026-02-17T12:35:36.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542351' 2026-02-17T12:35:36.829 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542351 2026-02-17T12:35:36.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:36.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542352 -lt 60129542351 2026-02-17T12:35:36.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:36.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:36.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:37.131 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:37.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:380: _scrub_abort: continue 2026-02-17T12:35:37.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:375: _scrub_abort: for i in $(seq 0 200) 2026-02-17T12:35:37.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:377: _scrub_abort: flush_pg_stats 2026-02-17T12:35:37.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:35:37.141 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:35:37.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:35:37.296 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:35:37.296 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:35:37.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:35:37.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:37.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:35:37.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836706 2026-02-17T12:35:37.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836706 2026-02-17T12:35:37.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836706' 2026-02-17T12:35:37.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:37.369 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:35:37.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673178 2026-02-17T12:35:37.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673178 2026-02-17T12:35:37.442 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836706 1-42949673178' 2026-02-17T12:35:37.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:35:37.443 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:35:37.514 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542355 2026-02-17T12:35:37.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542355 2026-02-17T12:35:37.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836706 1-42949673178 2-60129542355' 2026-02-17T12:35:37.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:37.515 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836706 2026-02-17T12:35:37.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:37.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:35:37.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836706 2026-02-17T12:35:37.517 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:37.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836706 2026-02-17T12:35:37.518 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836706 2026-02-17T12:35:37.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836706' 2026-02-17T12:35:37.519 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:37.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836703 -lt 21474836706 2026-02-17T12:35:37.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:35:38.677 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:35:38.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:35:38.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836706 -lt 21474836706 2026-02-17T12:35:38.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:38.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673178 2026-02-17T12:35:38.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:38.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:35:38.832 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673178 2026-02-17T12:35:38.832 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:38.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673178 2026-02-17T12:35:38.834 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673178 2026-02-17T12:35:38.834 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673178' 2026-02-17T12:35:38.834 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:35:38.991 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673178 -lt 42949673178 2026-02-17T12:35:38.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:35:38.992 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542355 2026-02-17T12:35:38.992 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:35:38.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:35:38.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542355 2026-02-17T12:35:38.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:35:38.995 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542355 2026-02-17T12:35:38.995 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542355 2026-02-17T12:35:38.996 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542355' 2026-02-17T12:35:38.996 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:35:39.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542355 -lt 60129542355 2026-02-17T12:35:39.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: ceph pg dump pgs 2026-02-17T12:35:39.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep '^1.0' 2026-02-17T12:35:39.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:378: _scrub_abort: grep -q scrubbing 2026-02-17T12:35:39.292 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:35:39.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:383: _scrub_abort: break 2026-02-17T12:35:39.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:385: _scrub_abort: set +o pipefail 2026-02-17T12:35:39.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:387: _scrub_abort: sleep 5 2026-02-17T12:35:44.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:389: _scrub_abort: grep 'noscrub set, aborting' td/osd-scrub-test/osd.1.log 2026-02-17T12:35:44.312 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T12:33:40.614+0000 7f2a09f89640 10 osd.1 pg_epoch: 18 pg[1.0( v 17'1000 (0'0,17'1000] local-lis/les=15/16 n=1000 ec=15/15 lis/c=15/15 les/c/f=16/16/0 sis=15) [1,0,2] r=0 lpr=15 crt=17'1000 lcod 17'999 mlcod 17'999 active+clean+scrubbing [ 1.0: ] ] scrubber: noscrub set, aborting 2026-02-17T12:35:44.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:395: _scrub_abort: get_last_scrub_stamp 1.0 2026-02-17T12:35:44.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:35:44.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:35:44.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:35:44.313 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:35:44.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:395: _scrub_abort: local last_scrub=2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:44.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:396: _scrub_abort: ceph config set osd osd_scrub_sleep 0.1 2026-02-17T12:35:44.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:398: _scrub_abort: ceph osd unset noscrub 2026-02-17T12:35:44.815 INFO:tasks.workunit.client.0.trial128.stderr:noscrub is unset 2026-02-17T12:35:44.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:399: _scrub_abort: '[' scrub = deep-scrub ']' 2026-02-17T12:35:44.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:403: _scrub_abort: TIMEOUT=500 2026-02-17T12:35:44.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:404: _scrub_abort: wait_for_scrub 1.0 2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:44.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T12:35:44.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:44.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:35:44.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:35:44.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:35:44.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:35:44.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:35:44.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:35:44.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:35:44.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:35:44.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:44.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:35:45.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:35:45.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:35:45.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:35:45.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:35:45.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:35:45.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:35:45.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:35:46.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:46.134 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:35:47.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:35:47.135 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:35:47.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:35:47.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:35:47.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:35:47.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:35:47.136 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:35:47.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:47.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:35:48.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:35:48.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:35:48.292 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:35:48.292 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:35:48.292 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:35:48.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:35:48.293 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:35:48.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:48.447 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:35:49.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:35:49.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:35:49.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:35:49.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:35:49.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:35:49.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:35:49.450 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:35:49.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:49.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:35:50.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:35:50.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:35:50.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:35:50.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:35:50.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:35:50.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:35:50.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:35:50.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:50.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:35:51.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:35:51.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:35:51.762 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:35:51.762 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:35:51.762 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:35:51.762 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:35:51.762 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:35:51.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:51.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:35:52.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:35:52.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:35:52.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:35:52.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:35:52.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:35:52.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:35:52.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:35:53.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:53.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:35:54.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:35:54.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:35:54.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:35:54.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:35:54.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:35:54.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:35:54.071 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:35:54.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:54.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:35:55.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:35:55.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:35:55.229 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:35:55.229 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:35:55.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:35:55.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:35:55.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:35:55.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:55.381 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:35:56.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:35:56.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:35:56.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:35:56.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:35:56.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:35:56.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:35:56.383 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:35:56.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:56.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:35:57.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:35:57.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:35:57.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:35:57.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:35:57.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:35:57.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:35:57.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:35:57.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:57.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:35:58.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:35:58.693 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:35:58.694 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:35:58.694 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:35:58.694 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:35:58.694 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:35:58.694 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:35:58.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:58.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:35:59.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:35:59.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:35:59.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:35:59.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:35:59.844 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:35:59.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:35:59.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:35:59.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:35:59.992 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:00.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:00.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:00.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:00.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:00.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:00.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:00.994 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:01.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:01.141 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:02.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:02.142 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:02.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:02.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:02.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:02.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:02.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:02.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:02.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:03.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:03.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:03.295 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:03.295 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:03.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:03.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:03.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:03.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:03.445 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:04.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:04.446 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:04.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:04.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:04.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:04.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:04.447 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:04.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:04.597 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:05.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:05.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:05.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:05.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:05.598 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:05.599 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:05.599 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:05.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:05.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:06.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:06.750 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:06.751 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:06.751 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:06.751 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:06.752 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:06.752 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:06.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:06.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:07.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:07.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:07.902 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:07.902 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:07.902 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:07.903 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:07.903 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:08.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:08.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:09.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:09.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:09.056 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:09.056 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:09.056 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:09.056 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:09.056 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:09.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:09.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:10.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:10.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:10.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:10.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:10.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:10.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:10.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:10.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:10.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:11.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:11.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:11.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:11.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:11.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:11.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:11.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:11.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:11.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:12.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:12.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:12.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:12.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:12.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:12.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:12.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:12.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:12.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:13.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:13.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:13.672 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:13.672 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:13.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:13.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:13.673 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:13.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:13.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:14.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:14.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:14.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:14.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:14.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:14.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:14.827 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:14.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:14.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:15.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:15.983 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:15.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:15.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:15.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:15.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:15.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:16.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:16.137 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:17.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:17.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:17.139 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:17.139 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:17.139 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:17.139 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:17.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:17.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:17.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:18.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:18.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:18.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:18.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:18.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:18.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:18.297 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:18.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:18.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:19.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:19.453 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:19.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:19.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:19.453 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:19.454 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:19.454 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:19.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:19.607 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:20.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:20.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:20.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:20.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:20.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:20.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:20.609 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:20.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:20.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:21.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:21.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:21.764 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:21.764 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:21.764 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:21.764 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:21.764 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:21.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:21.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:22.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:22.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:22.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:22.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:22.917 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:22.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:22.918 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:23.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:23.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:24.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:24.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:24.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:24.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:24.075 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:24.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:24.076 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:24.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:24.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:25.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:25.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:25.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:25.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:25.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:25.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:25.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:25.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:25.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:26.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:26.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:26.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:26.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:26.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:26.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:26.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:26.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:26.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:27.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:27.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:27.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:27.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:27.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:27.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:27.544 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:27.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:27.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:28.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:28.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:28.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:28.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:28.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:28.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:28.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:28.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:28.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:29.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:29.856 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:29.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:29.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:29.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:29.858 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:29.858 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:30.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:30.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:31.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:31.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:31.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:31.016 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:31.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:31.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:31.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:31.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:31.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:32.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:32.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:32.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:32.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:32.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:32.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:32.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:32.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:32.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:33.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:33.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:33.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:33.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:33.327 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:33.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:33.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:33.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:33.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:34.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:34.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:34.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:34.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:34.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:34.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:34.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:34.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:34.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:35.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:35.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:35.641 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:35.641 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:35.641 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:35.641 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:35.641 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:35.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:35.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:36.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:36.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:36.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:36.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:36.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:36.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:36.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:36.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:36.949 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:37.950 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:37.951 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:37.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:37.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:37.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:37.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:37.952 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:38.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:38.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:39.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:39.106 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:39.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:39.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:39.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:39.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:39.108 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:39.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:39.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:40.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:40.262 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:40.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:40.262 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:40.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:40.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:40.263 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:40.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:40.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:41.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:41.422 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:41.423 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:41.423 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:41.423 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:41.423 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:41.423 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:41.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:41.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:42.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:42.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:42.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:42.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:42.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:42.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:42.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:42.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-10T12:31:55.265558+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:42.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:36:43.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:36:43.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:36:43.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:36:43.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:36:43.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:36:43.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:36:43.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:36:43.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:36:41.939836+0000 '>' 2026-02-10T12:31:55.265558+0000 2026-02-17T12:36:43.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:36:43.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:405: _scrub_abort: perf_counters td/osd-scrub-test 3 2026-02-17T12:36:43.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-17T12:36:43.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-17T12:36:43.887 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-17T12:36:43.889 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-17T12:36:43.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:36:43.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-17T12:36:43.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.964 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:36:43.965 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:36:43.966 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:36:43.967 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:36:43.968 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:43.969 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:36:43.970 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:43.971 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:43.971 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:43.971 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:43.971 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:36:43.971 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:43.971 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:43.971 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:36:43.971 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:36:43.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:36:43.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-17T12:36:43.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:36:44.037 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:36:44.038 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:36:44.039 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.040 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 2, 2026-02-17T12:36:44.041 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 2, 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 1, 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 1, 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 55.07115783, 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 55.07115783 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 1, 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 1, 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 4.9999045, 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 4.9999045 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 11, 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:36:44.042 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 2, 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 2, 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 2 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:44.043 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:36:44.044 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:36:44.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:36:44.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-17T12:36:44.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:36:44.107 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:36:44.107 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:36:44.108 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:36:44.109 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.110 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:36:44.111 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:36:44.112 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:36:44.113 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:36:44.116 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.116 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.116 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.116 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.116 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:36:44.116 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:36:44.116 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:36:44.117 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:36:44.117 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:36:44.117 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:36:44.117 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:36:44.117 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:36:44.117 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:36:44.117 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:44.117 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:36:44.117 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:36:44.117 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:36:44.117 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Teardown Test TEST_scrub_abort ------------------ 2026-02-17T12:36:44.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_scrub_abort ------------------' 2026-02-17T12:36:44.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T12:36:44.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:36:44.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:36:44.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:36:44.118 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:36:44.118 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:36:44.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:36:44.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:36:44.118 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:36:44.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:36:44.227 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:36:44.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:36:44.228 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:36:44.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:36:44.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:36:44.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:36:44.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:36:44.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:36:44.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:36:44.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:36:44.233 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:36:44.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:36:44.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:36:44.288 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:36:44.289 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:36:44.289 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:36:44.289 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:36:44.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:36:44.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:36:44.290 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Complete Test TEST_scrub_abort ------------------ 2026-02-17T12:36:44.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_scrub_abort ------------------' 2026-02-17T12:36:44.290 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T12:36:44.290 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Prepare Test TEST_scrub_extended_sleep ------------------- 2026-02-17T12:36:44.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_scrub_extended_sleep -------------------' 2026-02-17T12:36:44.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T12:36:44.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T12:36:44.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T12:36:44.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:36:44.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:36:44.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:36:44.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:36:44.291 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:36:44.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:36:44.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:36:44.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:36:44.293 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:36:44.294 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:36:44.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:36:44.295 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:36:44.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:36:44.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:36:44.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:36:44.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:36:44.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:36:44.298 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:36:44.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:36:44.299 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:36:44.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:36:44.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:36:44.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:36:44.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:36:44.301 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:36:44.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:36:44.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:36:44.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:36:44.303 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T12:36:44.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:36:44.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:36:44.304 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:36:44.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.508089 2026-02-17T12:36:44.306 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:36:44.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:36:44.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:36:44.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T12:36:44.307 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Run Test TEST_scrub_extended_sleep ----------------------- 2026-02-17T12:36:44.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_scrub_extended_sleep -----------------------' 2026-02-17T12:36:44.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_scrub_extended_sleep td/osd-scrub-test 2026-02-17T12:36:44.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:209: TEST_scrub_extended_sleep: local dir=td/osd-scrub-test 2026-02-17T12:36:44.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:210: TEST_scrub_extended_sleep: local poolname=test 2026-02-17T12:36:44.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:211: TEST_scrub_extended_sleep: local OSDS=3 2026-02-17T12:36:44.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:212: TEST_scrub_extended_sleep: local objects=15 2026-02-17T12:36:44.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:214: TEST_scrub_extended_sleep: TESTDATA=testdata.508089 2026-02-17T12:36:44.307 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:216: TEST_scrub_extended_sleep: date +%w 2026-02-17T12:36:44.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:216: TEST_scrub_extended_sleep: DAY=2 2026-02-17T12:36:44.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:218: TEST_scrub_extended_sleep: '[' 2 -ge 4 ']' 2026-02-17T12:36:44.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:223: TEST_scrub_extended_sleep: expr 2 + 2 2026-02-17T12:36:44.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:223: TEST_scrub_extended_sleep: DAY_START=4 2026-02-17T12:36:44.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:224: TEST_scrub_extended_sleep: expr 2 + 3 2026-02-17T12:36:44.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:224: TEST_scrub_extended_sleep: DAY_END=5 2026-02-17T12:36:44.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:226: TEST_scrub_extended_sleep: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-17T12:36:44.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-test 2026-02-17T12:36:44.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:36:44.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:36:44.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:36:44.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-test/a 2026-02-17T12:36:44.312 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-17T12:36:44.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:36:44.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:36:44.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:36:44.332 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:36:44.332 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:36:44.332 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:36:44.333 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:36:44.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-17T12:36:44.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:36:44.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:36:44.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:36:44.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:36:44.363 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:36:44.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:36:44.364 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:36:44.364 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:36:44.364 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:36:44.365 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:36:44.365 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:36:44.365 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:36:44.365 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-mon.a.asok 2026-02-17T12:36:44.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:36:44.365 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.508089/ceph-mon.a.asok config get fsid 2026-02-17T12:36:44.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:36:44.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:36:44.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:36:44.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:36:44.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:36:44.420 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:36:44.421 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:36:44.421 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:36:44.421 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:36:44.421 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:36:44.421 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:36:44.421 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-mon.a.asok 2026-02-17T12:36:44.422 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:36:44.422 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.508089/ceph-mon.a.asok config get mon_host 2026-02-17T12:36:44.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:227: TEST_scrub_extended_sleep: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-17T12:36:44.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-test 2026-02-17T12:36:44.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:36:44.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:36:44.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:36:44.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-test/x 2026-02-17T12:36:44.477 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:36:44.567 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:36:44.567 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:36:44.567 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:36:44.567 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:36:44.567 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:36:44.568 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:36:44.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:36:44.568 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:36:44.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T12:36:44.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:229: TEST_scrub_extended_sleep: local 'ceph_osd_args=--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 ' 2026-02-17T12:36:44.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:230: TEST_scrub_extended_sleep: ceph_osd_args+='--osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 ' 2026-02-17T12:36:44.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:231: TEST_scrub_extended_sleep: ceph_osd_args+='--osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 ' 2026-02-17T12:36:44.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:232: TEST_scrub_extended_sleep: ceph_osd_args+='--osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 ' 2026-02-17T12:36:44.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:233: TEST_scrub_extended_sleep: ceph_osd_args+='--osd_op_queue=wpq --osd_scrub_end_week_day=5 ' 2026-02-17T12:36:44.587 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:234: TEST_scrub_extended_sleep: ceph_osd_args+=--bluestore_cache_autotune=false 2026-02-17T12:36:44.588 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:236: TEST_scrub_extended_sleep: expr 3 - 1 2026-02-17T12:36:44.588 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:236: TEST_scrub_extended_sleep: seq 0 2 2026-02-17T12:36:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:236: TEST_scrub_extended_sleep: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:36:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:238: TEST_scrub_extended_sleep: run_osd td/osd-scrub-test 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false 2026-02-17T12:36:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:36:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:36:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:36:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:36:44.592 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-17T12:36:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:36:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:36:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:36:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:36:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-17T12:36:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-17T12:36:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:36:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:36:44.593 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:36:44.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:36:44.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:36:44.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:36:44.594 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:36:44.594 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:36:44.594 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:36:44.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:36:44.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:36:44.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:36:44.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:36:44.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:36:44.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:36:44.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:36:44.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:36:44.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:36:44.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:36:44.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:36:44.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:36:44.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false' 2026-02-17T12:36:44.595 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-17T12:36:44.596 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:36:44.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=c612b083-4635-4603-8aa1-5f292c5db913 2026-02-17T12:36:44.596 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 c612b083-4635-4603-8aa1-5f292c5db913' 2026-02-17T12:36:44.596 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 c612b083-4635-4603-8aa1-5f292c5db913 2026-02-17T12:36:44.596 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:36:44.603 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDcYJRpcfmvIxAANmIspW3JB5/aPWHEwfcc5g== 2026-02-17T12:36:44.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDcYJRpcfmvIxAANmIspW3JB5/aPWHEwfcc5g=="}' 2026-02-17T12:36:44.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new c612b083-4635-4603-8aa1-5f292c5db913 -i td/osd-scrub-test/0/new.json 2026-02-17T12:36:44.695 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:36:44.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-17T12:36:44.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false --mkfs --key AQDcYJRpcfmvIxAANmIspW3JB5/aPWHEwfcc5g== --osd-uuid c612b083-4635-4603-8aa1-5f292c5db913 2026-02-17T12:36:44.721 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:44.711+0000 7fe902fb68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:36:44.722 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:44.715+0000 7fe902fb68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:36:44.723 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:44.715+0000 7fe902fb68c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:36:44.724 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:44.715+0000 7fe902fb68c0 -1 bdev(0x55c8f78cfc00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:36:44.724 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:44.715+0000 7fe902fb68c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T12:36:47.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-17T12:36:47.516 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:36:47.517 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:36:47.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:36:47.518 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:36:47.709 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:36:47.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:36:47.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false 2026-02-17T12:36:47.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:36:47.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:36:47.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:36:47.736 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:47.727+0000 7f7b555f98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:36:47.738 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:47.731+0000 7f7b555f98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:36:47.739 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:47.731+0000 7f7b555f98c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:36:47.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:36:47.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:36:47.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:36:47.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:36:47.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:36:47.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:36:47.867 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:36:47.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:36:47.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:36:47.867 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:36:48.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:36:49.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:36:49.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:36:49.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:36:49.017 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:36:49.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:36:49.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:36:49.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:36:49.247 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:49.239+0000 7f7b555f98c0 -1 Falling back to public interface 2026-02-17T12:36:50.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:36:50.169 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:36:50.170 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:36:50.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:36:50.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:36:50.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:36:50.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:36:51.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:36:51.326 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:36:51.327 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:36:51.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:36:51.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:36:51.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:36:51.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:36:52.254 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:52.247+0000 7f7b555f98c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:36:52.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:36:52.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:36:52.486 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:36:52.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:36:52.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:36:52.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:36:52.635 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1082238895,v1:127.0.0.1:6803/1082238895] [v2:127.0.0.1:6804/1082238895,v1:127.0.0.1:6805/1082238895] exists,up c612b083-4635-4603-8aa1-5f292c5db913 2026-02-17T12:36:52.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:36:52.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:36:52.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:36:52.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:236: TEST_scrub_extended_sleep: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:36:52.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:238: TEST_scrub_extended_sleep: run_osd td/osd-scrub-test 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false 2026-02-17T12:36:52.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:36:52.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:36:52.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:36:52.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:36:52.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-17T12:36:52.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:36:52.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:36:52.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:36:52.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:36:52.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-17T12:36:52.640 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-17T12:36:52.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:36:52.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:36:52.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:36:52.641 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:36:52.641 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:36:52.641 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:36:52.641 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:36:52.641 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:36:52.641 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:36:52.641 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:36:52.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:36:52.641 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:36:52.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:36:52.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:36:52.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:36:52.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:36:52.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:36:52.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:36:52.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:36:52.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:36:52.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:36:52.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false' 2026-02-17T12:36:52.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-17T12:36:52.642 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:36:52.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=758d9099-86c4-4fd8-a134-42b09b27abc6 2026-02-17T12:36:52.643 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 758d9099-86c4-4fd8-a134-42b09b27abc6 2026-02-17T12:36:52.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 758d9099-86c4-4fd8-a134-42b09b27abc6' 2026-02-17T12:36:52.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:36:52.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDkYJRpPd+WJhAAcLxV2RaMGB5rTvmqgWHWow== 2026-02-17T12:36:52.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDkYJRpPd+WJhAAcLxV2RaMGB5rTvmqgWHWow=="}' 2026-02-17T12:36:52.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 758d9099-86c4-4fd8-a134-42b09b27abc6 -i td/osd-scrub-test/1/new.json 2026-02-17T12:36:52.807 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:36:52.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-17T12:36:52.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false --mkfs --key AQDkYJRpPd+WJhAAcLxV2RaMGB5rTvmqgWHWow== --osd-uuid 758d9099-86c4-4fd8-a134-42b09b27abc6 2026-02-17T12:36:52.833 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:52.823+0000 7ff0389db8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:36:52.835 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:52.827+0000 7ff0389db8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:36:52.836 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:52.827+0000 7ff0389db8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:36:52.836 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:52.827+0000 7ff0389db8c0 -1 bdev(0x5566833ffc00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:36:52.836 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:52.827+0000 7ff0389db8c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T12:36:55.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-17T12:36:55.191 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:36:55.192 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:36:55.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:36:55.192 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:36:55.383 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:36:55.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:36:55.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false 2026-02-17T12:36:55.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:36:55.383 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:36:55.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:36:55.409 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:55.399+0000 7fe80f2e78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:36:55.411 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:55.403+0000 7fe80f2e78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:36:55.412 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:55.403+0000 7fe80f2e78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:36:55.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:36:55.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:36:55.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:36:55.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:36:55.538 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:36:55.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:36:55.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:36:55.539 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:36:55.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:36:55.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:36:55.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:36:56.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:36:56.694 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:36:56.694 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:36:56.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:36:56.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:36:56.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:36:56.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:36:56.927 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:56.919+0000 7fe80f2e78c0 -1 Falling back to public interface 2026-02-17T12:36:57.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:36:57.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:36:57.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:36:57.852 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:36:57.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:36:57.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:36:58.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:36:59.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:36:59.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:36:59.005 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:36:59.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:36:59.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:36:59.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:36:59.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:36:59.889 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:36:59.879+0000 7fe80f2e78c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:37:00.154 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:00.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:00.155 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:37:00.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:37:00.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:00.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:37:00.303 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/24478306,v1:127.0.0.1:6811/24478306] [v2:127.0.0.1:6812/24478306,v1:127.0.0.1:6813/24478306] exists,up 758d9099-86c4-4fd8-a134-42b09b27abc6 2026-02-17T12:37:00.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:37:00.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:37:00.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:37:00.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:236: TEST_scrub_extended_sleep: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:37:00.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:238: TEST_scrub_extended_sleep: run_osd td/osd-scrub-test 2 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false 2026-02-17T12:37:00.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:37:00.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:37:00.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:37:00.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:37:00.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-17T12:37:00.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:37:00.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:37:00.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:37:00.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:37:00.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-17T12:37:00.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-17T12:37:00.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:37:00.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:37:00.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:37:00.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:37:00.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:37:00.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:37:00.309 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:37:00.309 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:37:00.309 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:37:00.309 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:37:00.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:37:00.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:37:00.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:37:00.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:37:00.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:37:00.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:37:00.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:37:00.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:37:00.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:37:00.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:37:00.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:37:00.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false' 2026-02-17T12:37:00.310 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-17T12:37:00.310 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:37:00.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=415698c9-dd0a-430f-9bd5-537ce71e9ac8 2026-02-17T12:37:00.311 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 415698c9-dd0a-430f-9bd5-537ce71e9ac8 2026-02-17T12:37:00.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 415698c9-dd0a-430f-9bd5-537ce71e9ac8' 2026-02-17T12:37:00.312 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:37:00.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDsYJRp5PnJEhAA7K7ffJOXLS8oS2YTTjfdbQ== 2026-02-17T12:37:00.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDsYJRp5PnJEhAA7K7ffJOXLS8oS2YTTjfdbQ=="}' 2026-02-17T12:37:00.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 415698c9-dd0a-430f-9bd5-537ce71e9ac8 -i td/osd-scrub-test/2/new.json 2026-02-17T12:37:00.470 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:37:00.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-17T12:37:00.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false --mkfs --key AQDsYJRp5PnJEhAA7K7ffJOXLS8oS2YTTjfdbQ== --osd-uuid 415698c9-dd0a-430f-9bd5-537ce71e9ac8 2026-02-17T12:37:00.497 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:00.487+0000 7fe4f23678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:00.498 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:00.491+0000 7fe4f23678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:00.499 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:00.491+0000 7fe4f23678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:00.500 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:00.491+0000 7fe4f23678c0 -1 bdev(0x5565ce3ffc00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:37:00.500 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:00.491+0000 7fe4f23678c0 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-17T12:37:02.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-17T12:37:02.801 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:37:02.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:37:02.802 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T12:37:02.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:37:02.992 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T12:37:02.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:37:02.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --osd_scrub_sleep=0 --osd_scrub_extended_sleep=20 --osd_scrub_begin_week_day=4 --osd_op_queue=wpq --osd_scrub_end_week_day=5 --bluestore_cache_autotune=false 2026-02-17T12:37:02.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:37:02.993 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:37:02.994 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:37:03.018 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:03.011+0000 7f9f59dc38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:03.020 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:03.011+0000 7f9f59dc38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:03.021 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:03.011+0000 7f9f59dc38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:03.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:37:03.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:37:03.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:37:03.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:37:03.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:37:03.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:03.146 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:37:03.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:37:03.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:03.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:37:03.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:37:04.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:04.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:04.297 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:37:04.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:37:04.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:04.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:37:04.449 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:37:04.523 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:04.515+0000 7f9f59dc38c0 -1 Falling back to public interface 2026-02-17T12:37:05.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:05.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:05.451 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:37:05.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:37:05.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:05.451 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:37:05.598 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:37:06.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:06.600 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:06.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:37:06.601 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:37:06.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:06.602 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:37:06.753 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:37:07.522 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:07.515+0000 7f9f59dc38c0 -1 osd.2 0 log_to_monitors true 2026-02-17T12:37:07.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:07.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:07.755 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:37:07.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:37:07.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:07.756 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:37:07.904 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/724202841,v1:127.0.0.1:6819/724202841] [v2:127.0.0.1:6820/724202841,v1:127.0.0.1:6821/724202841] exists,up 415698c9-dd0a-430f-9bd5-537ce71e9ac8 2026-02-17T12:37:07.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:37:07.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:37:07.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:37:07.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:242: TEST_scrub_extended_sleep: create_pool test 1 1 2026-02-17T12:37:07.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T12:37:08.083 INFO:tasks.workunit.client.0.trial128.stderr:pool 'test' created 2026-02-17T12:37:08.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:37:09.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:243: TEST_scrub_extended_sleep: wait_for_clean 2026-02-17T12:37:09.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:37:09.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:37:09.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:37:09.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:37:09.096 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:37:09.096 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:37:09.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:37:09.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:37:09.097 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:37:09.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:37:09.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:37:09.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:37:09.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:37:09.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:37:09.203 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:37:09.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:37:09.360 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:37:09.360 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:37:09.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:37:09.360 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:37:09.361 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:37:09.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836498 2026-02-17T12:37:09.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836498 2026-02-17T12:37:09.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498' 2026-02-17T12:37:09.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:37:09.435 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:37:09.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705674 2026-02-17T12:37:09.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705674 2026-02-17T12:37:09.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705674' 2026-02-17T12:37:09.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:37:09.505 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:37:09.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542146 2026-02-17T12:37:09.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542146 2026-02-17T12:37:09.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-38654705674 2-60129542146' 2026-02-17T12:37:09.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:37:09.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836498 2026-02-17T12:37:09.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:37:09.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:37:09.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836498 2026-02-17T12:37:09.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:37:09.580 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836498 2026-02-17T12:37:09.580 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836498 2026-02-17T12:37:09.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836498' 2026-02-17T12:37:09.581 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:37:09.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836496 -lt 21474836498 2026-02-17T12:37:09.741 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:37:10.742 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:37:10.743 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:37:10.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836499 -lt 21474836498 2026-02-17T12:37:10.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:37:10.898 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705674 2026-02-17T12:37:10.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:37:10.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:37:10.900 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705674 2026-02-17T12:37:10.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:37:10.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705674 2026-02-17T12:37:10.902 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705674 2026-02-17T12:37:10.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-17T12:37:10.903 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:37:11.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705674 2026-02-17T12:37:11.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:37:11.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542146 2026-02-17T12:37:11.051 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:37:11.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:37:11.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542146 2026-02-17T12:37:11.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:37:11.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542146 2026-02-17T12:37:11.055 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542146 2026-02-17T12:37:11.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-17T12:37:11.055 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:37:11.212 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542148 -lt 60129542146 2026-02-17T12:37:11.212 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:37:11.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:37:11.213 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:37:11.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:37:11.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:37:11.400 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:37:11.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:37:11.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:37:11.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:37:11.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:37:11.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:37:11.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:37:11.555 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:37:11.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:37:11.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:37:11.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:37:11.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:37:11.736 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:37:11.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:246: TEST_scrub_extended_sleep: get_pg test SOMETHING 2026-02-17T12:37:11.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=test 2026-02-17T12:37:11.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T12:37:11.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map test SOMETHING 2026-02-17T12:37:11.737 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T12:37:11.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:246: TEST_scrub_extended_sleep: local pgid=1.0 2026-02-17T12:37:11.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:247: TEST_scrub_extended_sleep: get_primary test SOMETHING 2026-02-17T12:37:11.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T12:37:11.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T12:37:11.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test SOMETHING 2026-02-17T12:37:11.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:37:12.036 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:247: TEST_scrub_extended_sleep: local primary=1 2026-02-17T12:37:12.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:248: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:12.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:12.037 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:12.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:12.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:12.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:248: TEST_scrub_extended_sleep: local last_scrub=2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:12.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:249: TEST_scrub_extended_sleep: ceph tell 1.0 schedule-scrub 2026-02-17T12:37:12.248 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:37:12.249 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:37:12.249 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:37:12.249 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-10T12:35:32.244463+0000" 2026-02-17T12:37:12.249 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:37:12.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:252: TEST_scrub_extended_sleep: PASSED=false 2026-02-17T12:37:12.256 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:253: TEST_scrub_extended_sleep: (( i=0 )) 2026-02-17T12:37:12.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:253: TEST_scrub_extended_sleep: (( i < 15 )) 2026-02-17T12:37:12.257 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:254: TEST_scrub_extended_sleep: grep -q 'scrub state.*, sleeping' td/osd-scrub-test/osd.1.log 2026-02-17T12:37:12.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:259: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:13.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:253: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:13.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:253: TEST_scrub_extended_sleep: (( i < 15 )) 2026-02-17T12:37:13.259 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:254: TEST_scrub_extended_sleep: grep -q 'scrub state.*, sleeping' td/osd-scrub-test/osd.1.log 2026-02-17T12:37:13.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:256: TEST_scrub_extended_sleep: PASSED=true 2026-02-17T12:37:13.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:257: TEST_scrub_extended_sleep: break 2026-02-17T12:37:13.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:263: TEST_scrub_extended_sleep: '[' true = false ']' 2026-02-17T12:37:13.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:269: TEST_scrub_extended_sleep: ceph tell osd.1 config set osd_scrub_begin_week_day 0 2026-02-17T12:37:13.333 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:37:13.333 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_begin_week_day = '' (not observed, change may require restart) " 2026-02-17T12:37:13.333 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:37:13.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:270: TEST_scrub_extended_sleep: ceph tell osd.1 config set osd_scrub_end_week_day 0 2026-02-17T12:37:13.407 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:37:13.407 INFO:tasks.workunit.client.0.trial128.stdout: "success": "osd_scrub_end_week_day = '' (not observed, change may require restart) " 2026-02-17T12:37:13.408 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:37:13.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:274: TEST_scrub_extended_sleep: count=0 2026-02-17T12:37:13.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:275: TEST_scrub_extended_sleep: PASSED=false 2026-02-17T12:37:13.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i=0 )) 2026-02-17T12:37:13.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:13.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 0 + 1 2026-02-17T12:37:13.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=1 2026-02-17T12:37:13.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:13.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:13.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:13.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:13.418 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:13.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-17T12:37:08.076532+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:13.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:14.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:14.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:14.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 1 + 1 2026-02-17T12:37:14.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=2 2026-02-17T12:37:14.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:14.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:14.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:14.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:14.572 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:14.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-17T12:37:08.076532+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:14.724 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:15.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:15.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:15.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 2 + 1 2026-02-17T12:37:15.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=3 2026-02-17T12:37:15.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:15.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:15.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:15.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:15.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:15.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:15.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:16.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:16.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:16.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 3 + 1 2026-02-17T12:37:16.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=4 2026-02-17T12:37:16.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:16.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:16.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:16.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:16.887 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:17.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:17.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:18.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:18.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:18.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 4 + 1 2026-02-17T12:37:18.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=5 2026-02-17T12:37:18.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:18.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:18.042 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:18.042 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:18.042 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:18.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:18.196 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:19.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:19.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:19.198 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 5 + 1 2026-02-17T12:37:19.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=6 2026-02-17T12:37:19.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:19.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:19.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:19.200 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:19.201 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:19.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:19.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:20.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:20.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:20.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 6 + 1 2026-02-17T12:37:20.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=7 2026-02-17T12:37:20.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:20.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:20.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:20.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:20.360 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:20.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:20.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:21.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:21.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:21.510 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 7 + 1 2026-02-17T12:37:21.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=8 2026-02-17T12:37:21.512 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:21.512 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:21.512 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:21.512 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:21.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:21.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:21.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:22.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:22.672 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:22.672 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 8 + 1 2026-02-17T12:37:22.673 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=9 2026-02-17T12:37:22.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:22.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:22.674 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:22.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:22.675 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:22.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:22.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:23.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:23.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:23.828 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 9 + 1 2026-02-17T12:37:23.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=10 2026-02-17T12:37:23.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:23.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:23.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:23.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:23.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:23.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:23.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:24.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:24.986 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:24.987 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 10 + 1 2026-02-17T12:37:24.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=11 2026-02-17T12:37:24.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:24.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:24.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:24.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:24.989 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:25.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:25.138 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:26.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:26.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:26.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 11 + 1 2026-02-17T12:37:26.140 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=12 2026-02-17T12:37:26.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:26.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:26.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:26.142 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:26.143 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:26.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:26.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:27.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:27.296 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:27.296 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 12 + 1 2026-02-17T12:37:27.297 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=13 2026-02-17T12:37:27.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:27.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:27.298 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:27.299 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:27.299 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:27.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:27.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:28.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:28.448 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:28.449 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 13 + 1 2026-02-17T12:37:28.450 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=14 2026-02-17T12:37:28.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:28.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:28.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:28.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:28.451 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:28.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:28.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:29.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:29.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:29.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 14 + 1 2026-02-17T12:37:29.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=15 2026-02-17T12:37:29.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:29.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:29.607 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:29.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:29.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:29.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:29.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:30.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:30.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:30.767 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 15 + 1 2026-02-17T12:37:30.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=16 2026-02-17T12:37:30.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:30.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:30.769 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:30.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:30.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:30.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:30.917 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:31.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:31.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:31.919 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 16 + 1 2026-02-17T12:37:31.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=17 2026-02-17T12:37:31.920 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:31.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:31.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:31.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:31.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:32.076 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:32.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:33.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:33.077 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:33.077 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 17 + 1 2026-02-17T12:37:33.078 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=18 2026-02-17T12:37:33.079 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:33.079 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:33.079 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:33.079 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:33.079 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:33.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:33.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:34.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:34.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:34.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 18 + 1 2026-02-17T12:37:34.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=19 2026-02-17T12:37:34.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:34.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:34.233 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:34.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:34.234 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:34.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-10T12:35:32.244463+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:34.391 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:287: TEST_scrub_extended_sleep: sleep 1 2026-02-17T12:37:35.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i++ )) 2026-02-17T12:37:35.392 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:276: TEST_scrub_extended_sleep: (( i < 25 )) 2026-02-17T12:37:35.393 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: expr 19 + 1 2026-02-17T12:37:35.393 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:277: TEST_scrub_extended_sleep: count=20 2026-02-17T12:37:35.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: get_last_scrub_stamp 1.0 2026-02-17T12:37:35.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:37:35.394 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:37:35.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:37:35.395 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:37:35.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:278: TEST_scrub_extended_sleep: test 2026-02-17T12:37:32.827059+0000 '>' 2026-02-17T12:37:08.076532+0000 2026-02-17T12:37:35.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:280: TEST_scrub_extended_sleep: '[' 20 -lt 10 ']' 2026-02-17T12:37:35.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:284: TEST_scrub_extended_sleep: PASSED=true 2026-02-17T12:37:35.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:285: TEST_scrub_extended_sleep: break 2026-02-17T12:37:35.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:291: TEST_scrub_extended_sleep: '[' true = false ']' 2026-02-17T12:37:35.548 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Teardown Test TEST_scrub_extended_sleep ------------------ 2026-02-17T12:37:35.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_scrub_extended_sleep ------------------' 2026-02-17T12:37:35.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T12:37:35.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:37:35.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:37:35.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:37:35.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:37:35.550 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:37:35.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:37:35.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:37:35.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:37:35.857 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:37:35.858 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:37:35.859 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:37:35.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:37:35.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:37:35.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:37:35.862 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:37:35.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:37:35.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:37:35.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:37:35.864 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:37:35.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:37:35.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:37:35.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:37:35.873 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:37:35.873 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:37:35.873 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:37:35.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:37:35.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:37:35.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:37:35.875 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Complete Test TEST_scrub_extended_sleep ------------------ 2026-02-17T12:37:35.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_scrub_extended_sleep ------------------' 2026-02-17T12:37:35.876 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T12:37:35.876 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Prepare Test TEST_scrub_permit_time ------------------- 2026-02-17T12:37:35.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_scrub_permit_time -------------------' 2026-02-17T12:37:35.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T12:37:35.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T12:37:35.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T12:37:35.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:37:35.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:37:35.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:37:35.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:37:35.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:37:35.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:37:35.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:37:35.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:37:35.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:37:35.879 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:37:35.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:37:35.881 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:37:35.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:37:35.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:37:35.882 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:37:35.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:37:35.883 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:37:35.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:37:35.884 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:37:35.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:37:35.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:37:35.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:37:35.888 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:37:35.888 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:37:35.888 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:37:35.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:37:35.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:37:35.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:37:35.890 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T12:37:35.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:37:35.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:37:35.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:37:35.891 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.508089 2026-02-17T12:37:35.893 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:37:35.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:37:35.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:37:35.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T12:37:35.894 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Run Test TEST_scrub_permit_time ----------------------- 2026-02-17T12:37:35.895 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_scrub_permit_time -----------------------' 2026-02-17T12:37:35.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_scrub_permit_time td/osd-scrub-test 2026-02-17T12:37:35.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:419: TEST_scrub_permit_time: local dir=td/osd-scrub-test 2026-02-17T12:37:35.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:420: TEST_scrub_permit_time: local poolname=test 2026-02-17T12:37:35.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:421: TEST_scrub_permit_time: local OSDS=3 2026-02-17T12:37:35.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:422: TEST_scrub_permit_time: local objects=15 2026-02-17T12:37:35.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:424: TEST_scrub_permit_time: TESTDATA=testdata.508089 2026-02-17T12:37:35.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:426: TEST_scrub_permit_time: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-17T12:37:35.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-test 2026-02-17T12:37:35.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:37:35.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:37:35.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:37:35.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-test/a 2026-02-17T12:37:35.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-17T12:37:35.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:37:35.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:37:35.915 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:37:35.915 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:37:35.916 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:37:35.916 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:37:35.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:37:35.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-17T12:37:35.952 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:37:35.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:37:35.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:37:35.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:37:35.953 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:37:35.954 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:37:35.955 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:37:35.955 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:37:35.955 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:37:35.955 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:37:35.955 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:37:35.955 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:37:35.955 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-mon.a.asok 2026-02-17T12:37:35.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:37:35.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.508089/ceph-mon.a.asok config get fsid 2026-02-17T12:37:36.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:37:36.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:37:36.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:37:36.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:37:36.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:37:36.011 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:37:36.011 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:37:36.012 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:37:36.012 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:37:36.012 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:37:36.012 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:37:36.012 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-mon.a.asok 2026-02-17T12:37:36.012 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:37:36.012 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.508089/ceph-mon.a.asok config get mon_host 2026-02-17T12:37:36.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:427: TEST_scrub_permit_time: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-17T12:37:36.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-test 2026-02-17T12:37:36.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:37:36.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:37:36.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:37:36.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-test/x 2026-02-17T12:37:36.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:37:36.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:37:36.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:37:36.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:37:36.154 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:37:36.154 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:37:36.154 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:37:36.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:37:36.155 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:37:36.155 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T12:37:36.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:428: TEST_scrub_permit_time: date -d '2 hour ago' +%H 2026-02-17T12:37:36.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:428: TEST_scrub_permit_time: sed 's/^0//' 2026-02-17T12:37:36.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:428: TEST_scrub_permit_time: local scrub_begin_hour=10 2026-02-17T12:37:36.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:429: TEST_scrub_permit_time: date -d '1 hour ago' +%H 2026-02-17T12:37:36.175 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:429: TEST_scrub_permit_time: sed 's/^0//' 2026-02-17T12:37:36.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:429: TEST_scrub_permit_time: local scrub_end_hour=11 2026-02-17T12:37:36.178 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:430: TEST_scrub_permit_time: expr 3 - 1 2026-02-17T12:37:36.179 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:430: TEST_scrub_permit_time: seq 0 2 2026-02-17T12:37:36.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:430: TEST_scrub_permit_time: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:37:36.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:432: TEST_scrub_permit_time: run_osd td/osd-scrub-test 0 --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=10 --osd_scrub_end_hour=11 2026-02-17T12:37:36.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:37:36.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:37:36.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:37:36.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:37:36.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-17T12:37:36.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:37:36.183 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:37:36.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:37:36.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:37:36.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-17T12:37:36.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-17T12:37:36.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:37:36.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:37:36.184 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:37:36.184 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:37:36.184 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:37:36.184 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:37:36.184 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:37:36.184 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:37:36.185 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:37:36.185 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:37:36.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:37:36.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:37:36.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:37:36.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:37:36.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:37:36.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:37:36.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:37:36.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:37:36.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:37:36.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:37:36.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:37:36.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=10 --osd_scrub_end_hour=11' 2026-02-17T12:37:36.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-17T12:37:36.186 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:37:36.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a6dbaa87-ad28-4a4b-a512-035889e5d073 2026-02-17T12:37:36.186 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 a6dbaa87-ad28-4a4b-a512-035889e5d073' 2026-02-17T12:37:36.187 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 a6dbaa87-ad28-4a4b-a512-035889e5d073 2026-02-17T12:37:36.187 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:37:36.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAQYZRprghPCxAAwfldsSn7iT/5s8YO1sPlZw== 2026-02-17T12:37:36.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAQYZRprghPCxAAwfldsSn7iT/5s8YO1sPlZw=="}' 2026-02-17T12:37:36.195 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a6dbaa87-ad28-4a4b-a512-035889e5d073 -i td/osd-scrub-test/0/new.json 2026-02-17T12:37:36.283 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:37:36.291 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-17T12:37:36.292 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=10 --osd_scrub_end_hour=11 --mkfs --key AQAQYZRprghPCxAAwfldsSn7iT/5s8YO1sPlZw== --osd-uuid a6dbaa87-ad28-4a4b-a512-035889e5d073 2026-02-17T12:37:36.308 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:36.299+0000 7f801917b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:36.310 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:36.303+0000 7f801917b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:36.310 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:36.303+0000 7f801917b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:36.311 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:36.303+0000 7f801917b8c0 -1 bdev(0x55771da2fc00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:37:36.311 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:36.303+0000 7f801917b8c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T12:37:39.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-17T12:37:39.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:37:39.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:37:39.120 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:37:39.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:37:39.307 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:37:39.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:37:39.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=10 --osd_scrub_end_hour=11 2026-02-17T12:37:39.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:37:39.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:37:39.309 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:37:39.322 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:39.315+0000 7f074ecda8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:39.323 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:39.315+0000 7f074ecda8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:39.324 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:39.315+0000 7f074ecda8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:39.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:37:39.460 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:37:39.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:37:39.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:37:39.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:37:39.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:39.461 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:37:39.461 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:37:39.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:39.462 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:37:39.609 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:37:40.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:40.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:40.611 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:37:40.611 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:37:40.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:40.612 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:37:40.765 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:37:40.855 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:40.847+0000 7f074ecda8c0 -1 Falling back to public interface 2026-02-17T12:37:41.766 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:41.767 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:41.767 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:37:41.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:37:41.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:41.768 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:37:41.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:37:42.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:42.921 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:42.922 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:37:42.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:37:42.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:42.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:37:43.070 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:37:43.822 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:43.811+0000 7f074ecda8c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:37:44.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:44.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:44.073 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:37:44.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:37:44.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:44.075 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:37:44.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:37:45.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:45.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:45.230 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:37:45.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:37:45.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:45.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:37:45.380 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3736571961,v1:127.0.0.1:6803/3736571961] [v2:127.0.0.1:6804/3736571961,v1:127.0.0.1:6805/3736571961] exists,up a6dbaa87-ad28-4a4b-a512-035889e5d073 2026-02-17T12:37:45.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:37:45.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:37:45.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:37:45.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:430: TEST_scrub_permit_time: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:37:45.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:432: TEST_scrub_permit_time: run_osd td/osd-scrub-test 1 --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=10 --osd_scrub_end_hour=11 2026-02-17T12:37:45.384 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:37:45.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:37:45.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:37:45.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:37:45.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-17T12:37:45.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:37:45.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:37:45.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:37:45.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:37:45.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-17T12:37:45.385 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-17T12:37:45.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:37:45.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:37:45.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:37:45.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:37:45.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:37:45.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:37:45.386 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:37:45.386 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:37:45.386 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:37:45.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:37:45.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:37:45.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:37:45.386 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:37:45.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:37:45.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:37:45.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:37:45.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:37:45.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:37:45.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:37:45.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:37:45.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:37:45.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=10 --osd_scrub_end_hour=11' 2026-02-17T12:37:45.387 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-17T12:37:45.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:37:45.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ceff3ffa-95b1-4203-bf99-14a1678d4363 2026-02-17T12:37:45.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 ceff3ffa-95b1-4203-bf99-14a1678d4363' 2026-02-17T12:37:45.388 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 ceff3ffa-95b1-4203-bf99-14a1678d4363 2026-02-17T12:37:45.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:37:45.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAZYZRpnAZjFxAAruLsMMlvPBUgcJhh678YiA== 2026-02-17T12:37:45.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAZYZRpnAZjFxAAruLsMMlvPBUgcJhh678YiA=="}' 2026-02-17T12:37:45.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ceff3ffa-95b1-4203-bf99-14a1678d4363 -i td/osd-scrub-test/1/new.json 2026-02-17T12:37:45.549 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:37:45.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-17T12:37:45.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=10 --osd_scrub_end_hour=11 --mkfs --key AQAZYZRpnAZjFxAAruLsMMlvPBUgcJhh678YiA== --osd-uuid ceff3ffa-95b1-4203-bf99-14a1678d4363 2026-02-17T12:37:45.575 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:45.567+0000 7f177a5e88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:45.576 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:45.567+0000 7f177a5e88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:45.577 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:45.567+0000 7f177a5e88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:45.578 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:45.571+0000 7f177a5e88c0 -1 bdev(0x564e5b53fc00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:37:45.578 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:45.571+0000 7f177a5e88c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T12:37:47.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-17T12:37:47.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:37:47.903 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:37:47.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:37:47.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:37:48.091 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:37:48.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:37:48.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=10 --osd_scrub_end_hour=11 2026-02-17T12:37:48.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:37:48.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:37:48.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:37:48.115 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:48.106+0000 7fd5d1c298c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:48.116 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:48.106+0000 7fd5d1c298c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:48.117 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:48.106+0000 7fd5d1c298c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:48.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:37:48.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:37:48.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:37:48.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:37:48.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:37:48.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:48.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:37:48.250 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:37:48.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:48.251 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:37:48.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:37:49.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:49.405 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:49.406 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:37:49.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:37:49.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:49.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:37:49.559 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:37:49.627 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:49.618+0000 7fd5d1c298c0 -1 Falling back to public interface 2026-02-17T12:37:50.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:50.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:50.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:37:50.562 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:37:50.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:50.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:37:50.713 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:37:51.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:51.715 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:51.715 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:37:51.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:37:51.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:51.716 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:37:51.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:37:52.610 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:52.602+0000 7fd5d1c298c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:37:52.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:52.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:52.865 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:37:52.866 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:37:52.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:52.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:37:53.014 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:37:54.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:54.016 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:54.017 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:37:54.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:37:54.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:54.017 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:37:54.164 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/384938225,v1:127.0.0.1:6811/384938225] [v2:127.0.0.1:6812/384938225,v1:127.0.0.1:6813/384938225] exists,up ceff3ffa-95b1-4203-bf99-14a1678d4363 2026-02-17T12:37:54.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:37:54.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:37:54.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:37:54.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:430: TEST_scrub_permit_time: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:37:54.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:432: TEST_scrub_permit_time: run_osd td/osd-scrub-test 2 --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=10 --osd_scrub_end_hour=11 2026-02-17T12:37:54.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:37:54.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:37:54.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:37:54.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:37:54.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-17T12:37:54.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:37:54.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:37:54.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:37:54.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:37:54.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-17T12:37:54.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-17T12:37:54.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:37:54.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:37:54.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:37:54.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:37:54.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:37:54.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:37:54.168 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:37:54.168 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:37:54.168 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:37:54.169 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:37:54.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:37:54.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:37:54.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:37:54.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:37:54.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:37:54.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:37:54.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:37:54.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:37:54.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:37:54.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:37:54.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:37:54.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=10 --osd_scrub_end_hour=11' 2026-02-17T12:37:54.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-17T12:37:54.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:37:54.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=0f4234c8-fed8-44e3-9938-c609503cbb92 2026-02-17T12:37:54.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 0f4234c8-fed8-44e3-9938-c609503cbb92' 2026-02-17T12:37:54.172 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 0f4234c8-fed8-44e3-9938-c609503cbb92 2026-02-17T12:37:54.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:37:54.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAiYZRpFA2LChAA0Jxx27FyjEfOZLfEQ4NKZQ== 2026-02-17T12:37:54.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAiYZRpFA2LChAA0Jxx27FyjEfOZLfEQ4NKZQ=="}' 2026-02-17T12:37:54.182 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 0f4234c8-fed8-44e3-9938-c609503cbb92 -i td/osd-scrub-test/2/new.json 2026-02-17T12:37:54.332 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:37:54.342 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-17T12:37:54.343 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=10 --osd_scrub_end_hour=11 --mkfs --key AQAiYZRpFA2LChAA0Jxx27FyjEfOZLfEQ4NKZQ== --osd-uuid 0f4234c8-fed8-44e3-9938-c609503cbb92 2026-02-17T12:37:54.367 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:54.358+0000 7facc8a0e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:54.369 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:54.358+0000 7facc8a0e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:54.370 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:54.362+0000 7facc8a0e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:54.370 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:54.362+0000 7facc8a0e8c0 -1 bdev(0x55b67c717c00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:37:54.371 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:54.362+0000 7facc8a0e8c0 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-17T12:37:56.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-17T12:37:56.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:37:56.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:37:56.679 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T12:37:56.679 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:37:56.872 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T12:37:56.873 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:37:56.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --bluestore_cache_autotune=false --osd_deep_scrub_randomize_ratio=0.0 --osd_scrub_interval_randomize_ratio=0 --osd_scrub_begin_hour=10 --osd_scrub_end_hour=11 2026-02-17T12:37:56.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:37:56.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:37:56.874 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:37:56.888 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:56.878+0000 7efd149578c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:56.889 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:56.878+0000 7efd149578c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:56.890 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:56.882+0000 7efd149578c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:37:57.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:37:57.027 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:37:57.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:37:57.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:37:57.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:37:57.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:57.028 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:37:57.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:37:57.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:57.029 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:37:57.173 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:37:58.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:58.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:58.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:37:58.175 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:37:58.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:58.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:37:58.324 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:37:58.403 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:37:58.394+0000 7efd149578c0 -1 Falling back to public interface 2026-02-17T12:37:59.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:37:59.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:37:59.325 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:37:59.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:37:59.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:37:59.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:37:59.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:38:00.473 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:38:00.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:38:00.474 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:38:00.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:38:00.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:38:00.475 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:38:00.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:38:01.417 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:01.406+0000 7efd149578c0 -1 osd.2 0 log_to_monitors true 2026-02-17T12:38:01.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:38:01.626 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:38:01.627 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:38:01.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:38:01.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:38:01.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:38:01.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:38:02.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:38:02.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:38:02.795 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:38:02.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:38:02.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:38:02.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:38:02.941 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2716282351,v1:127.0.0.1:6819/2716282351] [v2:127.0.0.1:6820/2716282351,v1:127.0.0.1:6821/2716282351] exists,up 0f4234c8-fed8-44e3-9938-c609503cbb92 2026-02-17T12:38:02.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:38:02.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:38:02.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:38:02.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:440: TEST_scrub_permit_time: create_pool test 1 1 2026-02-17T12:38:02.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T12:38:03.134 INFO:tasks.workunit.client.0.trial128.stderr:pool 'test' created 2026-02-17T12:38:03.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:38:04.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:441: TEST_scrub_permit_time: wait_for_clean 2026-02-17T12:38:04.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:38:04.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:38:04.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:38:04.146 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:38:04.147 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:38:04.147 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:38:04.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:38:04.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:38:04.147 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:38:04.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:38:04.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:38:04.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:38:04.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:38:04.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:38:04.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:38:04.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:38:04.414 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:38:04.414 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:38:04.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:38:04.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:38:04.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:38:04.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836501 2026-02-17T12:38:04.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836501 2026-02-17T12:38:04.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501' 2026-02-17T12:38:04.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:38:04.487 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:38:04.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705676 2026-02-17T12:38:04.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705676 2026-02-17T12:38:04.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-38654705676' 2026-02-17T12:38:04.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:38:04.556 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:38:04.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542148 2026-02-17T12:38:04.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542148 2026-02-17T12:38:04.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-38654705676 2-60129542148' 2026-02-17T12:38:04.623 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:38:04.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836501 2026-02-17T12:38:04.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:38:04.624 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:38:04.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836501 2026-02-17T12:38:04.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:38:04.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836501 2026-02-17T12:38:04.627 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836501 2026-02-17T12:38:04.627 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836501' 2026-02-17T12:38:04.628 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:38:04.779 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836500 -lt 21474836501 2026-02-17T12:38:04.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:38:05.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:38:05.781 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:38:05.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836500 -lt 21474836501 2026-02-17T12:38:05.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:38:06.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-17T12:38:06.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:38:07.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836503 -lt 21474836501 2026-02-17T12:38:07.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:38:07.089 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705676 2026-02-17T12:38:07.089 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:38:07.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:38:07.091 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705676 2026-02-17T12:38:07.091 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:38:07.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705676 2026-02-17T12:38:07.093 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705676 2026-02-17T12:38:07.093 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705676' 2026-02-17T12:38:07.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:38:07.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705678 -lt 38654705676 2026-02-17T12:38:07.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:38:07.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542148 2026-02-17T12:38:07.247 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:38:07.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:38:07.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542148 2026-02-17T12:38:07.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:38:07.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542148 2026-02-17T12:38:07.250 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542148' 2026-02-17T12:38:07.250 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542148 2026-02-17T12:38:07.251 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:38:07.401 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542149 -lt 60129542148 2026-02-17T12:38:07.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:38:07.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:38:07.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:38:07.581 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:38:07.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:38:07.582 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:38:07.582 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:38:07.582 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:38:07.582 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:38:07.582 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:38:07.582 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:38:07.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:38:07.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:38:07.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:38:07.731 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:38:07.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:38:07.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:38:07.911 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:38:07.911 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:444: TEST_scrub_permit_time: get_pg test SOMETHING 2026-02-17T12:38:07.911 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=test 2026-02-17T12:38:07.911 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=SOMETHING 2026-02-17T12:38:07.911 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map test SOMETHING 2026-02-17T12:38:07.911 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-17T12:38:08.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:444: TEST_scrub_permit_time: local pgid=1.0 2026-02-17T12:38:08.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:445: TEST_scrub_permit_time: get_primary test SOMETHING 2026-02-17T12:38:08.060 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T12:38:08.060 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=SOMETHING 2026-02-17T12:38:08.060 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test SOMETHING 2026-02-17T12:38:08.060 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:38:08.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:445: TEST_scrub_permit_time: local primary=1 2026-02-17T12:38:08.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:446: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:08.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:08.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:08.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:08.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:08.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:446: TEST_scrub_permit_time: local last_scrub=2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:08.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:451: TEST_scrub_permit_time: ceph tell 1.0 schedule-scrub 86400 2026-02-17T12:38:08.417 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:38:08.417 INFO:tasks.workunit.client.0.trial128.stdout: "deep": false, 2026-02-17T12:38:08.417 INFO:tasks.workunit.client.0.trial128.stdout: "must": false, 2026-02-17T12:38:08.418 INFO:tasks.workunit.client.0.trial128.stdout: "stamp": "2026-02-16T12:36:28.413132+0000" 2026-02-17T12:38:08.418 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:38:08.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i=0 )) 2026-02-17T12:38:08.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:08.425 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:08.425 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:08.425 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:08.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:08.426 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:08.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-17T12:38:03.129224+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:08.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:09.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:09.579 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:09.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:09.579 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:09.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:09.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:09.580 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:09.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-17T12:38:03.129224+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:09.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:10.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:10.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:10.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:10.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:10.736 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:10.737 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:10.737 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:10.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:10.888 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:11.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:11.889 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:11.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:11.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:11.890 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:11.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:11.891 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:12.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:12.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:13.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:13.040 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:13.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:13.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:13.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:13.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:13.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:13.187 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:13.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:14.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:14.188 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:14.189 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:14.189 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:14.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:14.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:14.190 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:14.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:14.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:15.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:15.338 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:15.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:15.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:15.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:15.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:15.339 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:15.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:15.485 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:16.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:16.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:16.487 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:16.487 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:16.487 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:16.488 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:16.488 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:16.634 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:16.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:17.635 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:17.636 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:17.637 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:17.637 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:17.637 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:17.637 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:17.637 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:17.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:17.785 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:18.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:18.786 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:18.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:18.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:18.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:18.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:18.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:18.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:18.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:19.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:19.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:19.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:19.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:19.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:19.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:19.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:20.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:20.089 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:21.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:21.090 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:21.091 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:21.091 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:21.091 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:21.091 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:21.091 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:21.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:21.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:22.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:22.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:22.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:22.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:22.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:22.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:22.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:22.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:22.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:23.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:23.400 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:23.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:23.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:23.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:23.401 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:23.402 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:23.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:23.552 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:24.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:24.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:24.554 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:24.554 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:24.554 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:24.554 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:24.554 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:24.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:24.703 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:25.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:25.704 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:25.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:25.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:25.704 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:25.705 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:25.705 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:25.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:25.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:26.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:26.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:26.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:26.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:26.856 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:26.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:26.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:27.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:27.007 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:28.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:28.008 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:28.009 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:28.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:28.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:28.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:28.010 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:28.158 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:28.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:29.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:29.159 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:29.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:29.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:29.160 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:29.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:29.161 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:29.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:29.314 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:30.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:30.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:30.316 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:30.316 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:30.316 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:30.316 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:30.317 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:30.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:30.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:31.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:31.469 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:31.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:31.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:31.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:31.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:31.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:31.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:31.621 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:32.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:32.622 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:32.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:32.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:32.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:32.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:32.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:32.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:32.776 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:33.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:33.777 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:33.778 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:33.778 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:33.778 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:33.778 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:33.778 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:33.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:33.929 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:34.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:34.930 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:34.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:34.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:34.930 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:34.931 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:34.931 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:35.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:35.084 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:36.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:36.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:36.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:36.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:36.086 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:36.087 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:36.087 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:36.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:36.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:37.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:37.235 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:37.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:37.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:37.235 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:37.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:37.236 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:37.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:37.388 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:38.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:38.389 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:38.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:38.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:38.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:38.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:38.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:38.542 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:38.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:39.543 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:39.544 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:39.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:39.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:39.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:39.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:39.545 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:39.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:39.697 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:40.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:40.698 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:40.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:40.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:40.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:40.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:40.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:40.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:40.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:41.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:41.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:41.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: get_last_scrub_stamp 1.0 2026-02-17T12:38:41.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:38:41.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:38:41.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:38:41.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:38:42.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:455: TEST_scrub_permit_time: test 2026-02-16T12:36:28.413132+0000 '>' 2026-02-17T12:38:03.129224+0000 2026-02-17T12:38:42.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:458: TEST_scrub_permit_time: sleep 1 2026-02-17T12:38:43.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i++ )) 2026-02-17T12:38:43.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:454: TEST_scrub_permit_time: (( i < 30 )) 2026-02-17T12:38:43.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:460: TEST_scrub_permit_time: perf_counters td/osd-scrub-test 3 2026-02-17T12:38:43.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-17T12:38:43.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-17T12:38:43.006 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-17T12:38:43.007 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-17T12:38:43.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:38:43.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-17T12:38:43.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:38:43.072 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:38:43.073 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.074 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:38:43.075 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.076 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.076 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.076 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.076 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:38:43.076 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:38:43.076 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:38:43.076 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:38:43.076 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:38:43.076 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:38:43.076 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:38:43.078 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:38:43.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:38:43.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-17T12:38:43.079 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:38:43.140 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:38:43.140 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:38:43.140 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:38:43.140 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:38:43.141 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:38:43.141 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:38:43.141 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.141 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:38:43.141 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:38:43.141 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:38:43.141 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:38:43.141 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:38:43.142 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.142 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.142 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.142 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.142 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:38:43.142 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:38:43.142 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.142 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.143 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.143 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.143 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:38:43.143 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:38:43.143 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:38:43.143 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:38:43.143 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:38:43.143 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:38:43.144 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:38:43.144 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.144 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.144 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.144 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.144 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:38:43.144 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:38:43.144 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:38:43.145 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:38:43.145 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.145 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.145 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.145 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.145 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:38:43.145 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.145 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.146 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:38:43.146 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:38:43.146 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:38:43.146 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:38:43.146 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:38:43.146 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:38:43.146 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.146 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:38:43.147 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:38:43.147 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:38:43.147 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:38:43.147 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:38:43.147 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.147 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.147 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.147 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.148 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:38:43.148 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:38:43.148 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.148 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.148 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.148 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.148 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:38:43.148 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:38:43.148 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:38:43.149 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:38:43.149 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:38:43.149 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:38:43.149 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:38:43.149 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.149 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.149 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.149 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.150 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:38:43.150 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:38:43.150 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:38:43.150 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:38:43.150 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.150 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.150 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.150 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.151 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:38:43.151 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.151 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.151 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:38:43.151 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:38:43.151 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:38:43.151 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:38:43.151 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:38:43.151 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:38:43.152 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.152 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:38:43.152 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:38:43.152 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:38:43.152 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:38:43.152 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:38:43.152 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.152 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.153 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.153 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.153 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:38:43.153 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:38:43.153 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.153 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.153 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.153 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.154 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:38:43.154 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:38:43.154 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:38:43.154 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:38:43.154 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:38:43.154 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:38:43.154 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:38:43.154 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.155 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.155 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.155 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.155 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:38:43.155 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:38:43.155 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:38:43.155 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:38:43.155 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.155 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.156 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.156 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.156 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:38:43.156 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.156 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.156 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:38:43.156 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:38:43.156 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:38:43.157 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:38:43.157 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:38:43.157 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:38:43.157 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.157 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:38:43.157 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:38:43.157 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:38:43.157 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:38:43.157 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:38:43.158 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.158 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.158 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.158 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.158 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:38:43.158 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:38:43.158 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.158 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.159 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.159 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.159 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:38:43.159 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:38:43.159 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:38:43.159 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:38:43.159 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:38:43.159 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:38:43.159 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:38:43.160 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.160 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.160 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.161 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.161 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:38:43.161 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:38:43.161 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:38:43.161 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:38:43.161 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.161 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.161 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.162 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.162 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:38:43.162 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.162 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.162 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:38:43.162 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:38:43.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:38:43.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-17T12:38:43.163 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:38:43.212 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:38:43.212 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:38:43.212 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:38:43.212 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:38:43.213 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:38:43.213 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:38:43.213 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.213 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:38:43.213 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:38:43.213 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:38:43.213 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:38:43.213 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:38:43.213 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.213 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.213 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.213 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.214 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:38:43.214 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:38:43.214 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.214 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.214 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.214 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.214 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:38:43.214 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:38:43.214 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:38:43.214 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:38:43.214 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:38:43.214 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:38:43.214 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:38:43.215 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.215 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.215 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.215 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.215 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:38:43.215 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:38:43.215 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:38:43.215 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:38:43.215 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.215 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.215 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.215 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.216 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:38:43.216 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.216 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.216 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:38:43.216 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:38:43.216 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:38:43.216 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:38:43.216 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:38:43.216 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:38:43.216 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.216 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:38:43.216 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:38:43.217 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:38:43.217 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:38:43.217 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:38:43.217 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.217 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.217 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.217 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.217 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:38:43.217 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:38:43.217 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.217 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.217 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.217 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.218 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:38:43.218 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:38:43.218 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:38:43.218 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:38:43.218 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:38:43.218 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:38:43.218 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:38:43.218 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.218 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.218 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.218 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.218 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:38:43.218 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:38:43.219 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:38:43.219 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:38:43.219 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.219 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.219 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.219 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.219 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:38:43.219 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.219 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.219 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:38:43.219 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:38:43.219 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:38:43.219 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:38:43.220 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:38:43.220 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:38:43.220 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.220 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:38:43.220 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:38:43.220 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:38:43.220 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:38:43.220 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:38:43.220 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.220 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.220 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.220 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.220 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:38:43.221 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:38:43.221 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.221 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.221 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.221 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.221 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:38:43.221 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:38:43.221 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:38:43.221 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:38:43.221 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:38:43.221 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:38:43.221 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:38:43.221 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.222 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.222 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.222 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.222 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:38:43.222 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:38:43.222 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:38:43.222 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:38:43.222 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.222 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.222 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.222 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.222 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:38:43.222 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.223 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.223 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:38:43.223 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:38:43.223 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:38:43.223 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:38:43.223 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:38:43.223 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:38:43.223 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.223 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:38:43.223 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:38:43.223 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:38:43.223 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:38:43.223 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:38:43.224 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.224 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.224 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.224 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.224 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:38:43.224 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:38:43.224 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.224 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.224 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.224 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.224 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:38:43.224 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:38:43.224 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:38:43.225 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:38:43.225 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:38:43.225 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:38:43.225 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:38:43.225 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.225 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.226 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.226 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.226 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:38:43.226 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:38:43.226 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:38:43.226 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:38:43.226 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:38:43.226 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:38:43.226 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:38:43.226 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:38:43.226 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:38:43.226 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.227 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:38:43.227 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:38:43.227 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:38:43.227 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_scrub_permit_time ------------------' 2026-02-17T12:38:43.227 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Teardown Test TEST_scrub_permit_time ------------------ 2026-02-17T12:38:43.228 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T12:38:43.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:38:43.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:38:43.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:38:43.229 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:38:43.229 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:38:43.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:38:43.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:38:43.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:38:43.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:38:43.534 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:38:43.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:38:43.535 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:38:43.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:38:43.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:38:43.538 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:38:43.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:38:43.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:38:43.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:38:43.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:38:43.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:38:43.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:38:43.541 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:38:43.551 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:38:43.551 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:38:43.551 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:38:43.551 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:38:43.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:38:43.553 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:38:43.554 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Complete Test TEST_scrub_permit_time ------------------ 2026-02-17T12:38:43.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_scrub_permit_time ------------------' 2026-02-17T12:38:43.554 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:31: run: for func in $funcs 2026-02-17T12:38:43.555 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Prepare Test TEST_scrub_test ------------------- 2026-02-17T12:38:43.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:32: run: echo '-------------- Prepare Test TEST_scrub_test -------------------' 2026-02-17T12:38:43.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:33: run: setup td/osd-scrub-test 2026-02-17T12:38:43.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-scrub-test 2026-02-17T12:38:43.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-scrub-test 2026-02-17T12:38:43.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:38:43.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:38:43.556 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:38:43.557 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:38:43.557 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:38:43.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:38:43.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:38:43.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:38:43.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:38:43.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:38:43.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:38:43.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:38:43.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:38:43.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:38:43.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:38:43.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:38:43.561 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:38:43.562 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:38:43.562 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:38:43.563 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:38:43.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:38:43.564 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:38:43.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:38:43.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:38:43.565 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:38:43.565 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:38:43.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:38:43.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:38:43.567 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-scrub-test 2026-02-17T12:38:43.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:38:43.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:38:43.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:38:43.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.508089 2026-02-17T12:38:43.570 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:38:43.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-17T12:38:43.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:38:43.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-scrub-test 1' TERM HUP INT 2026-02-17T12:38:43.572 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Run Test TEST_scrub_test ----------------------- 2026-02-17T12:38:43.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:34: run: echo '-------------- Run Test TEST_scrub_test -----------------------' 2026-02-17T12:38:43.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:35: run: TEST_scrub_test td/osd-scrub-test 2026-02-17T12:38:43.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:52: TEST_scrub_test: local dir=td/osd-scrub-test 2026-02-17T12:38:43.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:53: TEST_scrub_test: local poolname=test 2026-02-17T12:38:43.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:54: TEST_scrub_test: local OSDS=3 2026-02-17T12:38:43.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:55: TEST_scrub_test: local objects=15 2026-02-17T12:38:43.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:57: TEST_scrub_test: TESTDATA=testdata.508089 2026-02-17T12:38:43.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:59: TEST_scrub_test: run_mon td/osd-scrub-test a --osd_pool_default_size=3 2026-02-17T12:38:43.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-scrub-test 2026-02-17T12:38:43.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:38:43.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:38:43.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:38:43.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-scrub-test/a 2026-02-17T12:38:43.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-scrub-test/a --run-dir=td/osd-scrub-test --osd_pool_default_size=3 2026-02-17T12:38:43.592 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:38:43.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:38:43.593 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:38:43.593 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:38:43.593 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:38:43.593 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:38:43.594 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:38:43.594 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-scrub-test/a '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --mon-cluster-log-file=td/osd-scrub-test/log --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=3 2026-02-17T12:38:43.620 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:38:43.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:38:43.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:38:43.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:38:43.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:38:43.621 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:38:43.622 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:38:43.622 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:38:43.622 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:38:43.622 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:38:43.623 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:38:43.623 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:38:43.623 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-mon.a.asok 2026-02-17T12:38:43.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:38:43.623 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.508089/ceph-mon.a.asok config get fsid 2026-02-17T12:38:43.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:38:43.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:38:43.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:38:43.678 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:38:43.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:38:43.679 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:38:43.679 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:38:43.679 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:38:43.679 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:38:43.679 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:38:43.679 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:38:43.680 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.508089/ceph-mon.a.asok 2026-02-17T12:38:43.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:38:43.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.508089/ceph-mon.a.asok config get mon_host 2026-02-17T12:38:43.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:60: TEST_scrub_test: run_mgr td/osd-scrub-test x --mgr_stats_period=1 2026-02-17T12:38:43.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-scrub-test 2026-02-17T12:38:43.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:38:43.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:38:43.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:38:43.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-scrub-test/x 2026-02-17T12:38:43.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:38:43.823 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:38:43.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:38:43.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:38:43.824 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:38:43.824 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:38:43.824 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:38:43.824 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:38:43.825 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:38:43.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-scrub-test/x '--log-file=td/osd-scrub-test/$name.log' '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --run-dir=td/osd-scrub-test '--pid-file=td/osd-scrub-test/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-17T12:38:43.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:61: TEST_scrub_test: local 'ceph_osd_args=--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 ' 2026-02-17T12:38:43.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:62: TEST_scrub_test: ceph_osd_args+='--osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 ' 2026-02-17T12:38:43.843 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:63: TEST_scrub_test: ceph_osd_args+=--osd_stats_update_period_scrubbing=2 2026-02-17T12:38:43.844 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:64: TEST_scrub_test: expr 3 - 1 2026-02-17T12:38:43.845 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:64: TEST_scrub_test: seq 0 2 2026-02-17T12:38:43.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:64: TEST_scrub_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:38:43.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:66: TEST_scrub_test: run_osd td/osd-scrub-test 0 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-17T12:38:43.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:38:43.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:38:43.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:38:43.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:38:43.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/0 2026-02-17T12:38:43.849 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:38:43.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:38:43.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:38:43.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:38:43.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/0' 2026-02-17T12:38:43.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/0/journal' 2026-02-17T12:38:43.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:38:43.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:38:43.850 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:38:43.850 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:38:43.850 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:38:43.851 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:38:43.851 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:38:43.851 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:38:43.851 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:38:43.851 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:38:43.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:38:43.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:38:43.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:38:43.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:38:43.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:38:43.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:38:43.851 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:38:43.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:38:43.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:38:43.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:38:43.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:38:43.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2026-02-17T12:38:43.852 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/0 2026-02-17T12:38:43.852 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:38:43.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=5727b570-940b-455f-a7cb-a8f309efc80c 2026-02-17T12:38:43.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 5727b570-940b-455f-a7cb-a8f309efc80c' 2026-02-17T12:38:43.853 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 5727b570-940b-455f-a7cb-a8f309efc80c 2026-02-17T12:38:43.853 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:38:43.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBTYZRpx2YaMxAADGWB/ePhQS4QPlAUbVXaug== 2026-02-17T12:38:43.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBTYZRpx2YaMxAADGWB/ePhQS4QPlAUbVXaug=="}' 2026-02-17T12:38:43.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 5727b570-940b-455f-a7cb-a8f309efc80c -i td/osd-scrub-test/0/new.json 2026-02-17T12:38:43.952 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:38:43.960 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/0/new.json 2026-02-17T12:38:43.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQBTYZRpx2YaMxAADGWB/ePhQS4QPlAUbVXaug== --osd-uuid 5727b570-940b-455f-a7cb-a8f309efc80c 2026-02-17T12:38:43.977 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:43.966+0000 7f687f0008c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:38:43.978 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:43.970+0000 7f687f0008c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:38:43.979 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:43.970+0000 7f687f0008c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:38:43.980 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:43.970+0000 7f687f0008c0 -1 bdev(0x55dc105f5c00 td/osd-scrub-test/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:38:43.980 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:43.970+0000 7f687f0008c0 -1 bluestore(td/osd-scrub-test/0) _read_fsid unparsable uuid 2026-02-17T12:38:46.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/0/keyring 2026-02-17T12:38:46.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:38:46.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:38:46.306 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:38:46.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:38:46.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:38:46.489 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:38:46.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/0 --osd-journal=td/osd-scrub-test/0/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-17T12:38:46.489 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:38:46.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:38:46.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:38:46.513 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:46.502+0000 7f43338278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:38:46.515 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:46.506+0000 7f43338278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:38:46.516 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:46.506+0000 7f43338278c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:38:46.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:38:46.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:38:46.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:38:46.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:38:46.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:38:46.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:38:46.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:38:46.643 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:38:46.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:38:46.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:38:46.790 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:38:47.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:38:47.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:38:47.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:38:47.792 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:38:47.792 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:38:47.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:38:47.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:38:48.011 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:48.001+0000 7f43338278c0 -1 Falling back to public interface 2026-02-17T12:38:48.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:38:48.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:38:48.940 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:38:48.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:38:48.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:38:48.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:38:49.086 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:38:50.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:38:50.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:38:50.088 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:38:50.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:38:50.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:38:50.088 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:38:50.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:38:51.012 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:51.005+0000 7f43338278c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:38:51.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:38:51.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:38:51.245 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:38:51.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:38:51.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:38:51.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:38:51.394 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:38:52.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:38:52.395 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:38:52.395 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:38:52.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:38:52.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:38:52.396 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:38:52.542 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1453966674,v1:127.0.0.1:6803/1453966674] [v2:127.0.0.1:6804/1453966674,v1:127.0.0.1:6805/1453966674] exists,up 5727b570-940b-455f-a7cb-a8f309efc80c 2026-02-17T12:38:52.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:38:52.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:38:52.546 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:38:52.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:64: TEST_scrub_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:38:52.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:66: TEST_scrub_test: run_osd td/osd-scrub-test 1 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-17T12:38:52.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:38:52.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:38:52.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:38:52.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:38:52.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/1 2026-02-17T12:38:52.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:38:52.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:38:52.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:38:52.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:38:52.547 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/1' 2026-02-17T12:38:52.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/1/journal' 2026-02-17T12:38:52.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:38:52.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:38:52.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:38:52.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:38:52.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:38:52.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:38:52.548 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:38:52.548 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:38:52.548 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:38:52.548 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:38:52.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:38:52.548 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:38:52.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:38:52.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:38:52.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:38:52.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:38:52.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:38:52.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:38:52.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:38:52.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:38:52.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:38:52.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2026-02-17T12:38:52.549 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/1 2026-02-17T12:38:52.549 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:38:52.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ea162911-3c11-4e3f-857b-78a000d098f1 2026-02-17T12:38:52.550 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 ea162911-3c11-4e3f-857b-78a000d098f1' 2026-02-17T12:38:52.551 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 ea162911-3c11-4e3f-857b-78a000d098f1 2026-02-17T12:38:52.551 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:38:52.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBcYZRpf14MIRAA7XZ8AhpEKBhQHP7aunDEtA== 2026-02-17T12:38:52.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBcYZRpf14MIRAA7XZ8AhpEKBhQHP7aunDEtA=="}' 2026-02-17T12:38:52.560 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ea162911-3c11-4e3f-857b-78a000d098f1 -i td/osd-scrub-test/1/new.json 2026-02-17T12:38:52.712 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:38:52.721 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/1/new.json 2026-02-17T12:38:52.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQBcYZRpf14MIRAA7XZ8AhpEKBhQHP7aunDEtA== --osd-uuid ea162911-3c11-4e3f-857b-78a000d098f1 2026-02-17T12:38:52.739 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:52.733+0000 7f811ece28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:38:52.741 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:52.733+0000 7f811ece28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:38:52.742 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:52.737+0000 7f811ece28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:38:52.742 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:52.737+0000 7f811ece28c0 -1 bdev(0x55a0b14d7c00 td/osd-scrub-test/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:38:52.742 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:52.737+0000 7f811ece28c0 -1 bluestore(td/osd-scrub-test/1) _read_fsid unparsable uuid 2026-02-17T12:38:55.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/1/keyring 2026-02-17T12:38:55.055 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:38:55.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:38:55.056 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:38:55.056 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:38:55.244 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:38:55.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:38:55.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/1 --osd-journal=td/osd-scrub-test/1/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-17T12:38:55.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:38:55.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:38:55.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:38:55.268 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:55.261+0000 7ff4509f88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:38:55.270 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:55.265+0000 7ff4509f88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:38:55.271 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:55.265+0000 7ff4509f88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:38:55.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:38:55.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:38:55.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:38:55.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:38:55.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:38:55.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:38:55.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:38:55.403 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:38:55.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:38:55.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:38:55.555 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:38:56.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:38:56.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:38:56.557 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:38:56.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:38:56.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:38:56.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:38:56.709 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:38:56.763 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:56.757+0000 7ff4509f88c0 -1 Falling back to public interface 2026-02-17T12:38:57.710 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:38:57.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:38:57.711 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:38:57.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:38:57.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:38:57.712 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:38:57.861 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:38:58.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:38:58.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:38:58.863 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:38:58.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:38:58.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:38:58.864 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:38:59.009 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:38:59.810 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:38:59.805+0000 7ff4509f88c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:39:00.010 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:39:00.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:39:00.011 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:39:00.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:39:00.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:00.011 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:39:00.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:39:01.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:39:01.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:39:01.166 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:39:01.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:39:01.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:01.166 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:39:01.315 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2906703138,v1:127.0.0.1:6811/2906703138] [v2:127.0.0.1:6812/2906703138,v1:127.0.0.1:6813/2906703138] exists,up ea162911-3c11-4e3f-857b-78a000d098f1 2026-02-17T12:39:01.318 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:39:01.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:39:01.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:39:01.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:64: TEST_scrub_test: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:39:01.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:66: TEST_scrub_test: run_osd td/osd-scrub-test 2 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-17T12:39:01.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-scrub-test 2026-02-17T12:39:01.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:39:01.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:39:01.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:39:01.319 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-scrub-test/2 2026-02-17T12:39:01.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:39:01.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:39:01.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:39:01.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:39:01.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-17T12:39:01.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-17T12:39:01.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:39:01.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:39:01.320 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:39:01.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:39:01.320 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:39:01.321 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:39:01.321 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:39:01.321 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:39:01.321 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:39:01.321 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:39:01.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:39:01.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:39:01.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:39:01.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:39:01.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:39:01.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:39:01.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:39:01.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:39:01.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:39:01.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:39:01.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:39:01.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2026-02-17T12:39:01.322 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-scrub-test/2 2026-02-17T12:39:01.322 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:39:01.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a9e90dc5-7473-4665-94ca-edb6ed294c04 2026-02-17T12:39:01.323 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 a9e90dc5-7473-4665-94ca-edb6ed294c04' 2026-02-17T12:39:01.323 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 a9e90dc5-7473-4665-94ca-edb6ed294c04 2026-02-17T12:39:01.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:39:01.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBlYZRpB/Z8ExAAR0ZEBuzgSwQ+HT9aF9AgMQ== 2026-02-17T12:39:01.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBlYZRpB/Z8ExAAR0ZEBuzgSwQ+HT9aF9AgMQ=="}' 2026-02-17T12:39:01.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a9e90dc5-7473-4665-94ca-edb6ed294c04 -i td/osd-scrub-test/2/new.json 2026-02-17T12:39:01.484 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:39:01.493 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-scrub-test/2/new.json 2026-02-17T12:39:01.494 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 --mkfs --key AQBlYZRpB/Z8ExAAR0ZEBuzgSwQ+HT9aF9AgMQ== --osd-uuid a9e90dc5-7473-4665-94ca-edb6ed294c04 2026-02-17T12:39:01.509 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:01.501+0000 7fd7f23298c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:39:01.510 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:01.505+0000 7fd7f23298c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:39:01.511 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:01.505+0000 7fd7f23298c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:39:01.512 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:01.505+0000 7fd7f23298c0 -1 bdev(0x55b1431c5c00 td/osd-scrub-test/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:39:01.512 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:01.505+0000 7fd7f23298c0 -1 bluestore(td/osd-scrub-test/2) _read_fsid unparsable uuid 2026-02-17T12:39:03.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-scrub-test/2/keyring 2026-02-17T12:39:03.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:39:03.812 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:39:03.813 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T12:39:03.813 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-scrub-test/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:39:04.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:39:04.002 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T12:39:04.002 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-17T12:39:04.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:39:04.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:39:04.003 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:39:04.026 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:04.021+0000 7fb4e47618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:39:04.028 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:04.021+0000 7fb4e47618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:39:04.029 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:04.021+0000 7fb4e47618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:39:04.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:39:04.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:39:04.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:39:04.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:39:04.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:39:04.156 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:39:04.156 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:39:04.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:39:04.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:04.157 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:39:04.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:39:05.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:39:05.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:39:05.315 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:39:05.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:39:05.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:05.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:39:05.465 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:39:05.503 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:05.497+0000 7fb4e47618c0 -1 Falling back to public interface 2026-02-17T12:39:06.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:39:06.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:39:06.466 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:39:06.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:39:06.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:06.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:39:06.613 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:39:07.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:39:07.615 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:39:07.616 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:39:07.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:39:07.616 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:07.617 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:39:07.761 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:39:08.533 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:08.525+0000 7fb4e47618c0 -1 osd.2 0 log_to_monitors true 2026-02-17T12:39:08.762 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:39:08.763 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:39:08.763 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:39:08.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:39:08.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:08.764 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:39:08.918 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:39:09.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:39:09.919 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:39:09.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:39:09.920 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:39:09.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:09.920 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:39:10.071 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1986722585,v1:127.0.0.1:6819/1986722585] [v2:127.0.0.1:6820/1986722585,v1:127.0.0.1:6821/1986722585] exists,up a9e90dc5-7473-4665-94ca-edb6ed294c04 2026-02-17T12:39:10.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:39:10.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:39:10.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:39:10.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:70: TEST_scrub_test: create_pool test 1 1 2026-02-17T12:39:10.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-17T12:39:10.267 INFO:tasks.workunit.client.0.trial128.stderr:pool 'test' created 2026-02-17T12:39:10.279 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-17T12:39:11.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:71: TEST_scrub_test: wait_for_clean 2026-02-17T12:39:11.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:39:11.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:39:11.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:39:11.282 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:39:11.282 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:39:11.282 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:39:11.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:39:11.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:39:11.283 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:39:11.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:39:11.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:39:11.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:39:11.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:39:11.382 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:39:11.382 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:39:11.539 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:39:11.539 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:39:11.540 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:39:11.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:39:11.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:11.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:39:11.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836501 2026-02-17T12:39:11.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836501 2026-02-17T12:39:11.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501' 2026-02-17T12:39:11.608 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:11.608 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:39:11.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672972 2026-02-17T12:39:11.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672972 2026-02-17T12:39:11.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672972' 2026-02-17T12:39:11.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:11.680 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:39:11.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=64424509444 2026-02-17T12:39:11.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 64424509444 2026-02-17T12:39:11.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-42949672972 2-64424509444' 2026-02-17T12:39:11.749 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:11.750 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836501 2026-02-17T12:39:11.750 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:11.751 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:39:11.752 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836501 2026-02-17T12:39:11.752 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:11.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836501 2026-02-17T12:39:11.754 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836501 2026-02-17T12:39:11.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836501' 2026-02-17T12:39:11.754 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:11.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836498 -lt 21474836501 2026-02-17T12:39:11.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:39:12.916 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:39:12.916 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:13.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836501 -lt 21474836501 2026-02-17T12:39:13.065 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:13.065 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672972 2026-02-17T12:39:13.065 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:13.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:39:13.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672972 2026-02-17T12:39:13.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:13.068 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672972 2026-02-17T12:39:13.069 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672972 2026-02-17T12:39:13.069 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672972' 2026-02-17T12:39:13.069 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:39:13.236 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672973 -lt 42949672972 2026-02-17T12:39:13.237 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:13.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-64424509444 2026-02-17T12:39:13.237 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:13.238 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:39:13.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-64424509444 2026-02-17T12:39:13.239 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:13.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=64424509444 2026-02-17T12:39:13.241 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 64424509444' 2026-02-17T12:39:13.241 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 64424509444 2026-02-17T12:39:13.241 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:39:13.390 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 64424509444 -lt 64424509444 2026-02-17T12:39:13.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:39:13.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:39:13.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:39:13.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:39:13.570 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:39:13.570 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:39:13.570 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:39:13.570 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:39:13.570 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:39:13.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:39:13.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:39:13.717 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:39:13.717 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:39:13.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:39:13.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:39:13.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:39:13.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:39:13.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:39:13.897 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:72: TEST_scrub_test: ceph osd dump 2026-02-17T12:39:13.897 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:72: TEST_scrub_test: grep '^pool.*['\'']test['\'']' 2026-02-17T12:39:13.897 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:72: TEST_scrub_test: awk '{ print $2 }' 2026-02-17T12:39:14.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:72: TEST_scrub_test: poolid=1 2026-02-17T12:39:14.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:74: TEST_scrub_test: dd if=/dev/urandom of=testdata.508089 bs=1032 count=1 2026-02-17T12:39:14.042 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records in 2026-02-17T12:39:14.043 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records out 2026-02-17T12:39:14.043 INFO:tasks.workunit.client.0.trial128.stderr:1032 bytes (1.0 kB, 1.0 KiB) copied, 3.224e-05 s, 32.0 MB/s 2026-02-17T12:39:14.043 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: seq 1 15 2026-02-17T12:39:14.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T12:39:14.043 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj1 testdata.508089 2026-02-17T12:39:14.062 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T12:39:14.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj2 testdata.508089 2026-02-17T12:39:14.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T12:39:14.085 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj3 testdata.508089 2026-02-17T12:39:14.109 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T12:39:14.110 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj4 testdata.508089 2026-02-17T12:39:14.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T12:39:14.136 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj5 testdata.508089 2026-02-17T12:39:14.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T12:39:14.161 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj6 testdata.508089 2026-02-17T12:39:14.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T12:39:14.185 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj7 testdata.508089 2026-02-17T12:39:14.209 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T12:39:14.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj8 testdata.508089 2026-02-17T12:39:14.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T12:39:14.234 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj9 testdata.508089 2026-02-17T12:39:14.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T12:39:14.258 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj10 testdata.508089 2026-02-17T12:39:14.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T12:39:14.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj11 testdata.508089 2026-02-17T12:39:14.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T12:39:14.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj12 testdata.508089 2026-02-17T12:39:14.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T12:39:14.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj13 testdata.508089 2026-02-17T12:39:14.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T12:39:14.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj14 testdata.508089 2026-02-17T12:39:14.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:75: TEST_scrub_test: for i in `seq 1 $objects` 2026-02-17T12:39:14.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:77: TEST_scrub_test: rados -p test put obj15 testdata.508089 2026-02-17T12:39:14.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:79: TEST_scrub_test: rm -f testdata.508089 2026-02-17T12:39:14.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:81: TEST_scrub_test: get_primary test obj1 2026-02-17T12:39:14.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T12:39:14.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T12:39:14.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T12:39:14.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:39:14.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:81: TEST_scrub_test: local primary=1 2026-02-17T12:39:14.559 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:82: TEST_scrub_test: get_not_primary test obj1 2026-02-17T12:39:14.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: get_not_primary: local poolname=test 2026-02-17T12:39:14.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1230: get_not_primary: local objectname=obj1 2026-02-17T12:39:14.560 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: get_primary test obj1 2026-02-17T12:39:14.560 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=test 2026-02-17T12:39:14.560 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=obj1 2026-02-17T12:39:14.561 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map test obj1 2026-02-17T12:39:14.561 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:39:14.710 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: get_not_primary: local primary=1 2026-02-17T12:39:14.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: get_not_primary: ceph --format json osd map test obj1 2026-02-17T12:39:14.711 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-17T12:39:14.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:82: TEST_scrub_test: local otherosd=0 2026-02-17T12:39:14.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:83: TEST_scrub_test: '[' 0 = 2 ']' 2026-02-17T12:39:14.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:87: TEST_scrub_test: local anotherosd=2 2026-02-17T12:39:14.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:90: TEST_scrub_test: CORRUPT_DATA=corrupt-data.508089 2026-02-17T12:39:14.866 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:91: TEST_scrub_test: dd if=/dev/urandom of=corrupt-data.508089 bs=512 count=1 2026-02-17T12:39:14.867 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records in 2026-02-17T12:39:14.867 INFO:tasks.workunit.client.0.trial128.stderr:1+0 records out 2026-02-17T12:39:14.867 INFO:tasks.workunit.client.0.trial128.stderr:512 bytes copied, 5.2808e-05 s, 9.7 MB/s 2026-02-17T12:39:14.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:92: TEST_scrub_test: objectstore_tool td/osd-scrub-test 2 obj1 set-bytes corrupt-data.508089 2026-02-17T12:39:14.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-scrub-test 2026-02-17T12:39:14.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:39:14.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T12:39:14.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:39:14.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-scrub-test 2 obj1 set-bytes corrupt-data.508089 2026-02-17T12:39:14.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-scrub-test 2026-02-17T12:39:14.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:39:14.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T12:39:14.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:39:14.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-scrub-test TERM osd.2 2026-02-17T12:39:14.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:39:14.868 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:39:14.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:39:14.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:39:14.868 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:39:16.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:39:16.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-scrub-test 2 obj1 set-bytes corrupt-data.508089 2026-02-17T12:39:16.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-scrub-test 2026-02-17T12:39:16.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:39:16.178 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T12:39:16.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:39:16.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-scrub-test/2 2026-02-17T12:39:16.179 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-scrub-test/2 obj1 set-bytes corrupt-data.508089 2026-02-17T12:39:17.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-scrub-test 2 --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-17T12:39:17.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-scrub-test 2026-02-17T12:39:17.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:39:17.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:39:17.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:39:17.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-scrub-test/2 2026-02-17T12:39:17.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 ' 2026-02-17T12:39:17.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:39:17.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:39:17.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:39:17.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-scrub-test/2' 2026-02-17T12:39:17.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-scrub-test/2/journal' 2026-02-17T12:39:17.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:39:17.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:39:17.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-scrub-test' 2026-02-17T12:39:17.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:39:17.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:39:17.355 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:39:17.355 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:39:17.355 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:39:17.355 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:39:17.357 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:39:17.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' 2026-02-17T12:39:17.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:39:17.357 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-scrub-test/$name.log' 2026-02-17T12:39:17.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-scrub-test/$name.pid' 2026-02-17T12:39:17.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:39:17.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:39:17.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:39:17.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:39:17.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:39:17.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+='--osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2' 2026-02-17T12:39:17.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-scrub-test/2 2026-02-17T12:39:17.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:39:17.358 INFO:tasks.workunit.client.0.trial128.stderr:start osd.2 2026-02-17T12:39:17.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=097bc8ee-5fd7-4d80-be8e-e460a7528c30 --auth-supported=none --mon-host=127.0.0.1:7138 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-scrub-test/2 --osd-journal=td/osd-scrub-test/2/journal --chdir= --run-dir=td/osd-scrub-test '--admin-socket=/tmp/ceph-asok.508089/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-scrub-test/$name.log' '--pid-file=td/osd-scrub-test/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-scrub-interval-randomize-ratio=0 --osd-deep-scrub-randomize-ratio=0 --osd_scrub_backoff_ratio=0 --osd_stats_update_period_not_scrubbing=3 --osd_stats_update_period_scrubbing=2 2026-02-17T12:39:17.359 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-scrub-test/2/whoami 2026-02-17T12:39:17.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:39:17.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:39:17.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:39:17.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:39:17.378 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:17.373+0000 7ff18c5ef8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:39:17.380 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:17.373+0000 7ff18c5ef8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:39:17.381 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:17.373+0000 7ff18c5ef8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:39:17.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:39:17.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:39:17.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:39:17.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:39:17.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:39:17.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:39:17.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:39:17.508 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:39:17.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:17.508 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:39:17.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:39:18.624 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:18.617+0000 7ff18c5ef8c0 -1 Falling back to public interface 2026-02-17T12:39:18.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:39:18.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:39:18.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:39:18.665 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:39:18.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:18.666 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:39:18.816 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:39:19.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:39:19.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:39:19.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:39:19.818 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:39:19.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:19.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:39:19.969 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:39:20.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:39:20.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:39:20.970 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:39:20.970 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:39:20.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:20.971 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:39:21.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:39:21.601 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:21.593+0000 7ff18c5ef8c0 -1 osd.2 19 log_to_monitors true 2026-02-17T12:39:22.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:39:22.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 500 )) 2026-02-17T12:39:22.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:39:22.122 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:39:22.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:22.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:39:22.275 INFO:tasks.workunit.client.0.trial128.stderr:osd.2 up in weight 1 up_from 23 up_thru 0 down_at 20 last_clean_interval [15,19) [v2:127.0.0.1:6818/2269114482,v1:127.0.0.1:6819/2269114482] [v2:127.0.0.1:6820/2269114482,v1:127.0.0.1:6821/2269114482] exists,up a9e90dc5-7473-4665-94ca-edb6ed294c04 2026-02-17T12:39:22.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:39:22.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:39:22.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:39:22.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:39:22.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:39:22.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:39:22.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:39:22.276 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:39:22.276 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:39:22.277 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:39:22.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:39:22.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:39:22.277 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:39:22.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:39:22.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:39:22.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:39:22.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:39:22.373 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:39:22.373 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:39:22.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:39:22.529 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:39:22.529 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:39:22.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:39:22.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:22.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:39:22.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836513 2026-02-17T12:39:22.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836513 2026-02-17T12:39:22.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513' 2026-02-17T12:39:22.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:22.601 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:39:22.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672984 2026-02-17T12:39:22.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672984 2026-02-17T12:39:22.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672984' 2026-02-17T12:39:22.671 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:22.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:39:22.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247811 2026-02-17T12:39:22.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247811 2026-02-17T12:39:22.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836513 1-42949672984 2-98784247811' 2026-02-17T12:39:22.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:22.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836513 2026-02-17T12:39:22.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:22.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:39:22.745 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836513 2026-02-17T12:39:22.745 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:22.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836513 2026-02-17T12:39:22.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836513' 2026-02-17T12:39:22.747 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836513 2026-02-17T12:39:22.747 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:22.897 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836511 -lt 21474836513 2026-02-17T12:39:22.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:39:23.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:39:23.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:24.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836511 -lt 21474836513 2026-02-17T12:39:24.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:39:25.054 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-17T12:39:25.054 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:25.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836514 -lt 21474836513 2026-02-17T12:39:25.206 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:25.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672984 2026-02-17T12:39:25.207 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:25.208 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:39:25.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672984 2026-02-17T12:39:25.209 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:25.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672984 2026-02-17T12:39:25.210 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672984' 2026-02-17T12:39:25.210 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 42949672984 2026-02-17T12:39:25.210 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:39:25.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672986 -lt 42949672984 2026-02-17T12:39:25.365 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:25.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-98784247811 2026-02-17T12:39:25.366 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:25.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:39:25.367 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-98784247811 2026-02-17T12:39:25.367 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:25.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247811 2026-02-17T12:39:25.369 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 98784247811' 2026-02-17T12:39:25.369 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 98784247811 2026-02-17T12:39:25.370 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:39:25.528 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247812 -lt 98784247811 2026-02-17T12:39:25.529 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:39:25.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:39:25.530 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:39:25.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:39:25.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:39:25.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:39:25.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:39:25.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:39:25.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:39:25.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:39:25.712 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:39:25.855 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:39:25.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:39:25.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:39:25.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:39:26.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:39:26.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:39:26.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:39:26.032 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:93: TEST_scrub_test: rm -f corrupt-data.508089 2026-02-17T12:39:26.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:95: TEST_scrub_test: local pgid=1.0 2026-02-17T12:39:26.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:96: TEST_scrub_test: pg_deep_scrub 1.0 2026-02-17T12:39:26.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: pg_deep_scrub: local pgid=1.0 2026-02-17T12:39:26.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: pg_deep_scrub: wait_for_pg_clean 1.0 2026-02-17T12:39:26.034 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=1.0 2026-02-17T12:39:26.034 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T12:39:26.034 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:39:26.035 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:39:26.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:39:26.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:39:26.035 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:39:26.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T12:39:26.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T12:39:26.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T12:39:26.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T12:39:26.321 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:39:26.321 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:39:26.466 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:39:26.467 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:39:26.467 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:39:26.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:39:26.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:26.467 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:39:26.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836518 2026-02-17T12:39:26.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836518 2026-02-17T12:39:26.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518' 2026-02-17T12:39:26.540 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:26.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:39:26.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672989 2026-02-17T12:39:26.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672989 2026-02-17T12:39:26.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-42949672989' 2026-02-17T12:39:26.610 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:26.611 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:39:26.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247815 2026-02-17T12:39:26.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247815 2026-02-17T12:39:26.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836518 1-42949672989 2-98784247815' 2026-02-17T12:39:26.684 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:26.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836518 2026-02-17T12:39:26.685 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:26.686 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:39:26.687 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836518 2026-02-17T12:39:26.687 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:26.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836518 2026-02-17T12:39:26.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836518' 2026-02-17T12:39:26.689 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836518 2026-02-17T12:39:26.689 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:26.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836516 -lt 21474836518 2026-02-17T12:39:26.853 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:39:27.854 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:39:27.855 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:28.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836516 -lt 21474836518 2026-02-17T12:39:28.005 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:39:29.006 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-17T12:39:29.006 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:29.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836519 -lt 21474836518 2026-02-17T12:39:29.162 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:29.163 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672989 2026-02-17T12:39:29.163 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:29.164 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:39:29.165 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672989 2026-02-17T12:39:29.165 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:29.167 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672989 2026-02-17T12:39:29.167 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672989 2026-02-17T12:39:29.168 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672989' 2026-02-17T12:39:29.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:39:29.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672991 -lt 42949672989 2026-02-17T12:39:29.325 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:29.326 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-98784247815 2026-02-17T12:39:29.326 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:29.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:39:29.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-98784247815 2026-02-17T12:39:29.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:29.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247815 2026-02-17T12:39:29.330 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 98784247815 2026-02-17T12:39:29.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 98784247815' 2026-02-17T12:39:29.331 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:39:29.483 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247817 -lt 98784247815 2026-02-17T12:39:29.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T12:39:29.484 INFO:tasks.workunit.client.0.trial128.stdout:#---------- 1.0 loop 0 2026-02-17T12:39:29.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T12:39:29.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T12:39:29.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=1.0 2026-02-17T12:39:29.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T12:39:29.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 1.0 query 2026-02-17T12:39:29.484 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T12:39:29.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean 2026-02-17T12:39:29.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T12:39:29.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T12:39:29.557 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T12:39:29.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-17T12:39:29.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:39:29.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:39:29.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:39:29.558 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-17T12:39:29.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: local last_scrub=2026-02-17T12:39:10.261798+0000 2026-02-17T12:39:29.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: pg_deep_scrub: ceph pg deep-scrub 1.0 2026-02-17T12:39:29.853 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 1.0 on osd.1 to deep-scrub 2026-02-17T12:39:29.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: pg_deep_scrub: wait_for_scrub 1.0 2026-02-17T12:39:10.261798+0000 last_deep_scrub_stamp 2026-02-17T12:39:29.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T12:39:29.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:39:10.261798+0000 2026-02-17T12:39:29.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-17T12:39:29.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:39:29.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:39:29.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-17T12:39:29.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:39:29.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:39:29.864 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:39:29.864 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-17T12:39:30.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:39:10.261798+0000 '>' 2026-02-17T12:39:10.261798+0000 2026-02-17T12:39:30.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:39:31.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:39:31.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:39:31.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-17T12:39:31.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:39:31.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:39:31.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-17T12:39:31.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:39:31.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:39:29.955674+0000 '>' 2026-02-17T12:39:10.261798+0000 2026-02-17T12:39:31.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:39:31.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:98: TEST_scrub_test: ceph pg dump pgs 2026-02-17T12:39:31.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:98: TEST_scrub_test: grep '^1.0' 2026-02-17T12:39:31.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:98: TEST_scrub_test: grep -q -- +inconsistent 2026-02-17T12:39:31.315 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:39:31.325 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:99: TEST_scrub_test: ceph pg 1.0 query 2026-02-17T12:39:31.326 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:99: TEST_scrub_test: jq .info.stats.stat_sum.num_scrub_errors 2026-02-17T12:39:31.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:99: TEST_scrub_test: test 2 = 2 2026-02-17T12:39:31.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:101: TEST_scrub_test: ceph osd out 1 2026-02-17T12:39:31.592 INFO:tasks.workunit.client.0.trial128.stderr:marked out osd.1. 2026-02-17T12:39:31.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:102: TEST_scrub_test: wait_for_clean 2026-02-17T12:39:31.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:39:31.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:39:31.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:39:31.605 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:39:31.606 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:39:31.606 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:39:31.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:39:31.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:39:31.606 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:39:31.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:39:31.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:39:31.726 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:39:31.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:39:31.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:39:31.727 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:39:31.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:39:31.875 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:39:31.875 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:39:31.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:39:31.875 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:31.876 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:39:31.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836524 2026-02-17T12:39:31.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836524 2026-02-17T12:39:31.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524' 2026-02-17T12:39:31.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:31.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:39:32.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672996 2026-02-17T12:39:32.019 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672996 2026-02-17T12:39:32.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-42949672996' 2026-02-17T12:39:32.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:32.020 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:39:32.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247822 2026-02-17T12:39:32.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247822 2026-02-17T12:39:32.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836524 1-42949672996 2-98784247822' 2026-02-17T12:39:32.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:32.092 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836524 2026-02-17T12:39:32.092 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:32.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:39:32.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836524 2026-02-17T12:39:32.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:32.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836524 2026-02-17T12:39:32.095 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836524' 2026-02-17T12:39:32.096 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836524 2026-02-17T12:39:32.096 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:32.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836524 -lt 21474836524 2026-02-17T12:39:32.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:32.245 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672996 2026-02-17T12:39:32.245 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:32.246 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:39:32.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672996 2026-02-17T12:39:32.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:32.248 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672996 2026-02-17T12:39:32.248 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672996 2026-02-17T12:39:32.249 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672996' 2026-02-17T12:39:32.249 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:39:32.402 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672996 -lt 42949672996 2026-02-17T12:39:32.403 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:32.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-98784247822 2026-02-17T12:39:32.403 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:32.404 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:39:32.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-98784247822 2026-02-17T12:39:32.405 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:32.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247822 2026-02-17T12:39:32.407 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 98784247822' 2026-02-17T12:39:32.407 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 98784247822 2026-02-17T12:39:32.408 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:39:32.558 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247822 -lt 98784247822 2026-02-17T12:39:32.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:39:32.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:39:32.560 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:39:32.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:39:32.735 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:39:32.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:39:32.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:39:32.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:39:32.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:39:32.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:39:32.735 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:39:32.885 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:39:32.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:39:32.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:39:32.886 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:39:33.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:39:33.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:39:33.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:39:33.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:104: TEST_scrub_test: pg_deep_scrub 1.0 2026-02-17T12:39:33.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: pg_deep_scrub: local pgid=1.0 2026-02-17T12:39:33.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: pg_deep_scrub: wait_for_pg_clean 1.0 2026-02-17T12:39:33.067 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: local pg_id=1.0 2026-02-17T12:39:33.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-17T12:39:33.068 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:39:33.068 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:39:33.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:39:33.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:39:33.068 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:39:33.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-17T12:39:33.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1712: wait_for_pg_clean: local -a delays 2026-02-17T12:39:33.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1713: wait_for_pg_clean: local -i loop=0 2026-02-17T12:39:33.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1715: wait_for_pg_clean: flush_pg_stats 2026-02-17T12:39:33.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:39:33.349 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:39:33.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:39:33.506 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:39:33.506 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:39:33.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:39:33.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:33.507 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:39:33.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836527 2026-02-17T12:39:33.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836527 2026-02-17T12:39:33.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527' 2026-02-17T12:39:33.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:33.576 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:39:33.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949672998 2026-02-17T12:39:33.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949672998 2026-02-17T12:39:33.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-42949672998' 2026-02-17T12:39:33.646 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:33.646 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:39:33.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247825 2026-02-17T12:39:33.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247825 2026-02-17T12:39:33.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836527 1-42949672998 2-98784247825' 2026-02-17T12:39:33.718 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:33.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836527 2026-02-17T12:39:33.719 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:33.720 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:39:33.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836527 2026-02-17T12:39:33.721 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:33.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836527 2026-02-17T12:39:33.722 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836527 2026-02-17T12:39:33.722 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836527' 2026-02-17T12:39:33.723 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:33.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836524 -lt 21474836527 2026-02-17T12:39:33.886 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:39:34.887 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:39:34.888 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:35.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836527 -lt 21474836527 2026-02-17T12:39:35.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:35.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949672998 2026-02-17T12:39:35.038 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:35.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:39:35.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949672998 2026-02-17T12:39:35.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:35.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949672998 2026-02-17T12:39:35.042 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949672998' 2026-02-17T12:39:35.042 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949672998 2026-02-17T12:39:35.042 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:39:35.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949672999 -lt 42949672998 2026-02-17T12:39:35.198 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:35.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-98784247825 2026-02-17T12:39:35.199 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:35.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:39:35.201 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-98784247825 2026-02-17T12:39:35.201 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:35.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247825 2026-02-17T12:39:35.203 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 98784247825 2026-02-17T12:39:35.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 98784247825' 2026-02-17T12:39:35.203 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:39:35.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247825 -lt 98784247825 2026-02-17T12:39:35.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: true 2026-02-17T12:39:35.362 INFO:tasks.workunit.client.0.trial128.stdout:#---------- 1.0 loop 0 2026-02-17T12:39:35.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1718: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-17T12:39:35.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: is_pg_clean 1.0 2026-02-17T12:39:35.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1581: is_pg_clean: local pgid=1.0 2026-02-17T12:39:35.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1582: is_pg_clean: local pg_state 2026-02-17T12:39:35.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: ceph pg 1.0 query 2026-02-17T12:39:35.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: jq -r '.state ' 2026-02-17T12:39:35.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1583: is_pg_clean: pg_state=active+clean+remapped 2026-02-17T12:39:35.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1584: is_pg_clean: [[ active+clean+remapped == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-17T12:39:35.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: break 2026-02-17T12:39:35.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1728: wait_for_pg_clean: return 0 2026-02-17T12:39:35.434 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-17T12:39:35.435 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:39:35.435 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:39:35.435 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:39:35.435 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-17T12:39:35.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: pg_deep_scrub: local last_scrub=2026-02-17T12:39:29.955674+0000 2026-02-17T12:39:35.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: pg_deep_scrub: ceph pg deep-scrub 1.0 2026-02-17T12:39:35.723 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 1.0 on osd.0 to deep-scrub 2026-02-17T12:39:35.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: pg_deep_scrub: wait_for_scrub 1.0 2026-02-17T12:39:29.955674+0000 last_deep_scrub_stamp 2026-02-17T12:39:35.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T12:39:35.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:39:29.955674+0000 2026-02-17T12:39:35.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_deep_scrub_stamp 2026-02-17T12:39:35.733 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:39:35.734 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:39:35.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-17T12:39:35.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:39:35.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:39:35.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:39:35.734 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-17T12:39:35.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:39:29.955674+0000 '>' 2026-02-17T12:39:29.955674+0000 2026-02-17T12:39:35.882 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:39:36.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:39:36.884 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:39:36.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-17T12:39:36.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:39:36.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:39:36.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:39:36.885 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-17T12:39:37.037 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:39:29.955674+0000 '>' 2026-02-17T12:39:29.955674+0000 2026-02-17T12:39:37.038 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:39:38.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:39:38.039 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:39:38.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_deep_scrub_stamp 2026-02-17T12:39:38.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:39:38.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_deep_scrub_stamp 2026-02-17T12:39:38.040 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:39:38.041 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_deep_scrub_stamp' 2026-02-17T12:39:38.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:39:36.216238+0000 '>' 2026-02-17T12:39:29.955674+0000 2026-02-17T12:39:38.194 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:39:38.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:106: TEST_scrub_test: ceph pg 1.0 query 2026-02-17T12:39:38.195 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:106: TEST_scrub_test: jq .info.stats.stat_sum.num_scrub_errors 2026-02-17T12:39:38.267 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:106: TEST_scrub_test: test 2 = 2 2026-02-17T12:39:38.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:107: TEST_scrub_test: ceph pg 1.0 query 2026-02-17T12:39:38.268 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:107: TEST_scrub_test: jq '.peer_info[0].stats.stat_sum.num_scrub_errors' 2026-02-17T12:39:38.339 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:107: TEST_scrub_test: test 2 = 2 2026-02-17T12:39:38.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:108: TEST_scrub_test: ceph pg dump pgs 2026-02-17T12:39:38.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:108: TEST_scrub_test: grep '^1.0' 2026-02-17T12:39:38.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:108: TEST_scrub_test: grep -q -- +inconsistent 2026-02-17T12:39:38.475 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:39:38.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:110: TEST_scrub_test: ceph osd in 1 2026-02-17T12:39:38.678 INFO:tasks.workunit.client.0.trial128.stderr:marked in osd.1. 2026-02-17T12:39:38.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:111: TEST_scrub_test: wait_for_clean 2026-02-17T12:39:38.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:39:38.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:39:38.687 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:39:38.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:39:38.688 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:39:38.688 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:39:38.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:39:38.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:39:38.688 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:39:38.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:39:38.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:39:38.806 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:39:38.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:39:38.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:39:38.807 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:39:38.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:39:38.957 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:39:38.957 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:39:38.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:39:38.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:38.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:39:39.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836533 2026-02-17T12:39:39.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836533 2026-02-17T12:39:39.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533' 2026-02-17T12:39:39.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:39.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:39:39.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673005 2026-02-17T12:39:39.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673005 2026-02-17T12:39:39.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-42949673005' 2026-02-17T12:39:39.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:39.102 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:39:39.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247831 2026-02-17T12:39:39.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247831 2026-02-17T12:39:39.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836533 1-42949673005 2-98784247831' 2026-02-17T12:39:39.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:39.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836533 2026-02-17T12:39:39.172 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:39.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:39:39.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836533 2026-02-17T12:39:39.174 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:39.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836533 2026-02-17T12:39:39.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836533' 2026-02-17T12:39:39.175 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836533 2026-02-17T12:39:39.176 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:39.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836531 -lt 21474836533 2026-02-17T12:39:39.332 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:39:40.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:39:40.333 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:40.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836534 -lt 21474836533 2026-02-17T12:39:40.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:40.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673005 2026-02-17T12:39:40.491 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:40.492 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:39:40.493 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673005 2026-02-17T12:39:40.494 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:40.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673005 2026-02-17T12:39:40.495 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673005' 2026-02-17T12:39:40.495 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673005 2026-02-17T12:39:40.495 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:39:40.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673006 -lt 42949673005 2026-02-17T12:39:40.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:40.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-98784247831 2026-02-17T12:39:40.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:40.649 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:39:40.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-98784247831 2026-02-17T12:39:40.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:40.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247831 2026-02-17T12:39:40.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 98784247831' 2026-02-17T12:39:40.652 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 98784247831 2026-02-17T12:39:40.653 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:39:40.815 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247832 -lt 98784247831 2026-02-17T12:39:40.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:39:40.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:39:40.816 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:39:41.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:39:41.004 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:39:41.004 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:39:41.004 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:39:41.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:39:41.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:39:41.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:39:41.005 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:39:41.153 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:39:41.153 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:39:41.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:39:41.154 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:39:41.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:39:41.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:39:41.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:39:41.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:113: TEST_scrub_test: repair 1.0 2026-02-17T12:39:41.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1896: repair: local pgid=1.0 2026-02-17T12:39:41.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: get_last_scrub_stamp 1.0 2026-02-17T12:39:41.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:39:41.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:39:41.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:39:41.336 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:39:41.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1897: repair: local last_scrub=2026-02-17T12:39:36.216238+0000 2026-02-17T12:39:41.490 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1898: repair: ceph pg repair 1.0 2026-02-17T12:39:41.629 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 1.0 on osd.1 to repair 2026-02-17T12:39:41.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1899: repair: wait_for_scrub 1.0 2026-02-17T12:39:36.216238+0000 2026-02-17T12:39:41.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2072: wait_for_scrub: local pgid=1.0 2026-02-17T12:39:41.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2073: wait_for_scrub: local last_scrub=2026-02-17T12:39:36.216238+0000 2026-02-17T12:39:41.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2074: wait_for_scrub: local sname=last_scrub_stamp 2026-02-17T12:39:41.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i=0 )) 2026-02-17T12:39:41.639 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:39:41.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:39:41.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:39:41.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:39:41.640 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:39:41.640 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:39:41.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:39:36.216238+0000 '>' 2026-02-17T12:39:36.216238+0000 2026-02-17T12:39:41.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2080: wait_for_scrub: sleep 1 2026-02-17T12:39:42.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i++ )) 2026-02-17T12:39:42.796 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2076: wait_for_scrub: (( i < 500 )) 2026-02-17T12:39:42.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-17T12:39:42.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: get_last_scrub_stamp: local pgid=1.0 2026-02-17T12:39:42.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-17T12:39:42.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-17T12:39:42.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-17T12:39:42.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2077: wait_for_scrub: test 2026-02-17T12:39:41.951222+0000 '>' 2026-02-17T12:39:36.216238+0000 2026-02-17T12:39:42.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2078: wait_for_scrub: return 0 2026-02-17T12:39:42.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:114: TEST_scrub_test: wait_for_clean 2026-02-17T12:39:42.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:39:42.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:39:42.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:39:42.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:39:42.947 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:39:42.947 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:39:42.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:39:42.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:39:42.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:39:43.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:39:43.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:39:43.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:39:43.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:39:43.053 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:39:43.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:39:43.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:39:43.204 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:39:43.204 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:39:43.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:39:43.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:43.204 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:39:43.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836539 2026-02-17T12:39:43.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836539 2026-02-17T12:39:43.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836539' 2026-02-17T12:39:43.276 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:43.276 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:39:43.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673010 2026-02-17T12:39:43.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673010 2026-02-17T12:39:43.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836539 1-42949673010' 2026-02-17T12:39:43.347 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:43.347 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:39:43.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247836 2026-02-17T12:39:43.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247836 2026-02-17T12:39:43.419 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836539 1-42949673010 2-98784247836' 2026-02-17T12:39:43.420 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:43.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836539 2026-02-17T12:39:43.420 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:43.421 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:39:43.422 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836539 2026-02-17T12:39:43.422 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:43.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836539 2026-02-17T12:39:43.424 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836539 2026-02-17T12:39:43.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836539' 2026-02-17T12:39:43.424 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:43.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836536 -lt 21474836539 2026-02-17T12:39:43.582 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:39:44.583 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:39:44.584 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:44.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836539 -lt 21474836539 2026-02-17T12:39:44.738 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:44.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673010 2026-02-17T12:39:44.739 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:44.740 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:39:44.741 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673010 2026-02-17T12:39:44.741 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:44.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673010 2026-02-17T12:39:44.743 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673010' 2026-02-17T12:39:44.743 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673010 2026-02-17T12:39:44.744 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:39:44.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673011 -lt 42949673010 2026-02-17T12:39:44.896 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:44.897 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-98784247836 2026-02-17T12:39:44.897 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:44.898 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:39:44.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-98784247836 2026-02-17T12:39:44.899 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:44.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247836 2026-02-17T12:39:44.901 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 98784247836 2026-02-17T12:39:44.901 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 98784247836' 2026-02-17T12:39:44.901 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:39:45.066 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247837 -lt 98784247836 2026-02-17T12:39:45.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:39:45.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:39:45.067 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:39:45.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:39:45.255 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:39:45.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:39:45.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:39:45.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:39:45.255 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:39:45.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:39:45.256 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:39:45.406 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:39:45.406 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:39:45.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:39:45.407 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:39:45.588 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:39:45.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:39:45.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:39:45.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:117: TEST_scrub_test: ceph pg 1.0 query 2026-02-17T12:39:45.589 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:117: TEST_scrub_test: jq '.peer_info[0].stats.stat_sum.num_scrub_errors' 2026-02-17T12:39:45.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:117: TEST_scrub_test: test 2 = 2 2026-02-17T12:39:45.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:118: TEST_scrub_test: ceph pg dump pgs 2026-02-17T12:39:45.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:118: TEST_scrub_test: grep '^1.0' 2026-02-17T12:39:45.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:118: TEST_scrub_test: grep -vq -- +inconsistent 2026-02-17T12:39:45.793 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:39:45.802 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:120: TEST_scrub_test: ceph osd out 1 2026-02-17T12:39:46.002 INFO:tasks.workunit.client.0.trial128.stderr:marked out osd.1. 2026-02-17T12:39:46.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:121: TEST_scrub_test: wait_for_clean 2026-02-17T12:39:46.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:39:46.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:39:46.015 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:39:46.015 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:39:46.016 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:39:46.016 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:39:46.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:39:46.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:39:46.017 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:39:46.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:39:46.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:39:46.119 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:39:46.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:39:46.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=500 2026-02-17T12:39:46.120 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:39:46.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:39:46.272 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:39:46.272 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:39:46.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:39:46.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:46.272 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:39:46.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836543 2026-02-17T12:39:46.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836543 2026-02-17T12:39:46.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836543' 2026-02-17T12:39:46.344 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:46.344 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:39:46.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=42949673014 2026-02-17T12:39:46.414 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 42949673014 2026-02-17T12:39:46.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836543 1-42949673014' 2026-02-17T12:39:46.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:39:46.415 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:39:46.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=98784247840 2026-02-17T12:39:46.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 98784247840 2026-02-17T12:39:46.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836543 1-42949673014 2-98784247840' 2026-02-17T12:39:46.487 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:46.488 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836543 2026-02-17T12:39:46.488 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:46.488 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:39:46.489 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836543 2026-02-17T12:39:46.490 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:46.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836543 2026-02-17T12:39:46.491 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836543' 2026-02-17T12:39:46.491 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836543 2026-02-17T12:39:46.492 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:46.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836541 -lt 21474836543 2026-02-17T12:39:46.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:39:47.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 500 -eq 0 ']' 2026-02-17T12:39:47.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:47.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836541 -lt 21474836543 2026-02-17T12:39:47.803 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:39:48.804 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 499 -eq 0 ']' 2026-02-17T12:39:48.805 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:39:48.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836544 -lt 21474836543 2026-02-17T12:39:48.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:48.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-42949673014 2026-02-17T12:39:48.955 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:48.956 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:39:48.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-42949673014 2026-02-17T12:39:48.957 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:48.958 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=42949673014 2026-02-17T12:39:48.959 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 42949673014 2026-02-17T12:39:48.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 42949673014' 2026-02-17T12:39:48.959 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:39:49.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 42949673016 -lt 42949673014 2026-02-17T12:39:49.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:39:49.121 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-98784247840 2026-02-17T12:39:49.121 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:39:49.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:39:49.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-98784247840 2026-02-17T12:39:49.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:39:49.124 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=98784247840 2026-02-17T12:39:49.125 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 98784247840' 2026-02-17T12:39:49.125 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 98784247840 2026-02-17T12:39:49.125 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:39:49.285 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 98784247842 -lt 98784247840 2026-02-17T12:39:49.286 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:39:49.286 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:39:49.286 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:39:49.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:39:49.471 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:39:49.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:39:49.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:39:49.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:39:49.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:39:49.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:39:49.472 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:39:49.625 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:39:49.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:39:49.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:39:49.626 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:39:49.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:39:49.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:39:49.807 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:39:49.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:123: TEST_scrub_test: ceph pg 1.0 query 2026-02-17T12:39:49.808 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:123: TEST_scrub_test: jq .info.stats.stat_sum.num_scrub_errors 2026-02-17T12:39:49.877 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:123: TEST_scrub_test: test 0 = 0 2026-02-17T12:39:49.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:124: TEST_scrub_test: ceph pg 1.0 query 2026-02-17T12:39:49.878 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:124: TEST_scrub_test: jq '.peer_info[0].stats.stat_sum.num_scrub_errors' 2026-02-17T12:39:49.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:124: TEST_scrub_test: test 0 = 0 2026-02-17T12:39:49.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:125: TEST_scrub_test: ceph pg 1.0 query 2026-02-17T12:39:49.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:125: TEST_scrub_test: jq '.peer_info[1].stats.stat_sum.num_scrub_errors' 2026-02-17T12:39:50.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:125: TEST_scrub_test: test 0 = 0 2026-02-17T12:39:50.020 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:126: TEST_scrub_test: ceph pg dump pgs 2026-02-17T12:39:50.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:126: TEST_scrub_test: grep '^1.0' 2026-02-17T12:39:50.021 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:126: TEST_scrub_test: grep -vq -- +inconsistent 2026-02-17T12:39:50.162 INFO:tasks.workunit.client.0.trial128.stderr:dumped pgs 2026-02-17T12:39:50.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:127: TEST_scrub_test: perf_counters td/osd-scrub-test 3 2026-02-17T12:39:50.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:43: perf_counters: local dir=td/osd-scrub-test 2026-02-17T12:39:50.172 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:44: perf_counters: local OSDS=3 2026-02-17T12:39:50.172 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: expr 3 - 1 2026-02-17T12:39:50.173 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: seq 0 2 2026-02-17T12:39:50.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:39:50.174 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.0 counter dump 2026-02-17T12:39:50.175 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:39:50.239 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:39:50.239 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:39:50.239 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:39:50.239 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:39:50.239 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:39:50.239 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:39:50.239 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.239 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:39:50.239 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:39:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:39:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:39:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:39:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.240 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:39:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:39:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.240 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.241 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:39:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:39:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:39:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:39:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:39:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:39:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:39:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.241 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.241 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:39:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:39:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:39:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:39:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.242 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:39:50.242 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.242 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.242 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:39:50.242 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:39:50.243 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:39:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:39:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:39:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:39:50.243 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:39:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 1, 2026-02-17T12:39:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 1, 2026-02-17T12:39:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 1, 2026-02-17T12:39:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:39:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 1, 2026-02-17T12:39:50.243 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0.003999932, 2026-02-17T12:39:50.244 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0.003999932 2026-02-17T12:39:50.244 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.244 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:39:50.244 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:39:50.244 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.244 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.244 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.244 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.244 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:39:50.244 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 2, 2026-02-17T12:39:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:39:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:39:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:39:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:39:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:39:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.245 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:39:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:39:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 1, 2026-02-17T12:39:50.245 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:39:50.246 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.246 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.246 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.246 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.246 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 2 2026-02-17T12:39:50.246 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.246 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.246 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:39:50.246 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:39:50.246 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:39:50.246 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:39:50.246 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:39:50.246 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:39:50.247 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.247 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:39:50.247 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:39:50.247 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:39:50.247 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:39:50.247 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:39:50.247 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.247 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.247 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.247 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.247 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:39:50.247 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:39:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.248 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:39:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:39:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:39:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:39:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:39:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:39:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:39:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.248 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.249 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:39:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:39:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:39:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:39:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.249 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.249 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:39:50.249 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.249 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.250 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:39:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:39:50.250 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:39:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:39:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:39:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:39:50.250 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:39:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:39:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:39:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:39:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:39:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.250 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.251 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:39:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:39:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.251 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:39:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:39:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:39:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:39:50.251 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:39:50.252 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:39:50.252 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:39:50.252 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.252 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.252 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.253 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.253 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:39:50.253 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:39:50.253 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:39:50.253 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:39:50.253 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.253 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.253 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.253 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.253 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:39:50.253 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.253 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.254 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:39:50.254 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:39:50.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:39:50.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.1 counter dump 2026-02-17T12:39:50.254 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:39:50.309 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:39:50.309 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:39:50.309 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:39:50.309 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:39:50.309 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:39:50.309 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:39:50.309 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:39:50.310 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 3, 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 3, 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 3, 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 3, 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0.011999791, 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0.00399993 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.311 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 6, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 3, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 2 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:39:50.312 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:39:50.313 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.314 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:39:50.315 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:39:50.315 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:39:50.315 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:39:50.315 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:39:50.315 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:39:50.315 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:39:50.316 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:45: perf_counters: for osd in $(seq 0 $(expr $OSDS - 1)) 2026-02-17T12:39:50.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: ceph tell osd.2 counter dump 2026-02-17T12:39:50.317 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:47: perf_counters: jq 'with_entries(select(.key | startswith("osd_scrub")))' 2026-02-17T12:39:50.381 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:39:50.382 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_ec": [ 2026-02-17T12:39:50.382 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:39:50.382 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:39:50.382 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:39:50.382 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:39:50.382 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.382 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:39:50.382 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:39:50.382 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:39:50.382 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:39:50.383 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:39:50.383 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.383 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.383 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.383 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.383 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:39:50.383 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:39:50.383 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.383 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.383 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.383 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.383 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:39:50.383 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:39:50.384 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:39:50.384 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:39:50.384 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:39:50.384 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:39:50.384 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:39:50.384 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.384 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.384 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.384 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.384 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:39:50.384 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:39:50.384 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:39:50.385 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:39:50.385 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.385 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.385 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.385 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.385 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:39:50.385 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.385 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.385 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:39:50.385 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_dp_repl": [ 2026-02-17T12:39:50.385 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:39:50.385 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:39:50.385 INFO:tasks.workunit.client.0.trial128.stdout: "level": "deep", 2026-02-17T12:39:50.386 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:39:50.386 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.386 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:39:50.386 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:39:50.386 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:39:50.386 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:39:50.386 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:39:50.386 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.386 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.386 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.386 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.386 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:39:50.387 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:39:50.387 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.387 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.387 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.387 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.387 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:39:50.387 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:39:50.387 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:39:50.387 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:39:50.387 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:39:50.387 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:39:50.387 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:39:50.388 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.388 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.388 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.388 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.388 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:39:50.388 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:39:50.388 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:39:50.388 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:39:50.388 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.388 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.388 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.388 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.389 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:39:50.389 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.389 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.389 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:39:50.389 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_ec": [ 2026-02-17T12:39:50.389 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:39:50.389 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:39:50.389 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:39:50.389 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "ec" 2026-02-17T12:39:50.389 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.389 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:39:50.389 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:39:50.390 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:39:50.390 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:39:50.390 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:39:50.390 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.390 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.390 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.390 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.390 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:39:50.390 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:39:50.390 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.390 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.390 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.391 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.391 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:39:50.391 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:39:50.391 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:39:50.391 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:39:50.391 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:39:50.391 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:39:50.391 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:39:50.391 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.391 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.391 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.391 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.391 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:39:50.392 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:39:50.392 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:39:50.392 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:39:50.392 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.392 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.392 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.392 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.392 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:39:50.392 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.392 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.392 INFO:tasks.workunit.client.0.trial128.stdout: ], 2026-02-17T12:39:50.392 INFO:tasks.workunit.client.0.trial128.stdout: "osd_scrub_sh_repl": [ 2026-02-17T12:39:50.392 INFO:tasks.workunit.client.0.trial128.stdout: { 2026-02-17T12:39:50.393 INFO:tasks.workunit.client.0.trial128.stdout: "labels": { 2026-02-17T12:39:50.393 INFO:tasks.workunit.client.0.trial128.stdout: "level": "shallow", 2026-02-17T12:39:50.393 INFO:tasks.workunit.client.0.trial128.stdout: "pooltype": "replicated" 2026-02-17T12:39:50.393 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.393 INFO:tasks.workunit.client.0.trial128.stdout: "counters": { 2026-02-17T12:39:50.393 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_started": 0, 2026-02-17T12:39:50.393 INFO:tasks.workunit.client.0.trial128.stdout: "num_scrubs_past_reservation": 0, 2026-02-17T12:39:50.393 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs": 0, 2026-02-17T12:39:50.393 INFO:tasks.workunit.client.0.trial128.stdout: "successful_scrubs_elapsed": { 2026-02-17T12:39:50.393 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.393 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.393 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.394 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.394 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs": 0, 2026-02-17T12:39:50.394 INFO:tasks.workunit.client.0.trial128.stdout: "failed_scrubs_elapsed": { 2026-02-17T12:39:50.394 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.394 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.394 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.394 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.394 INFO:tasks.workunit.client.0.trial128.stdout: "preemptions": 0, 2026-02-17T12:39:50.394 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_selected": 0, 2026-02-17T12:39:50.394 INFO:tasks.workunit.client.0.trial128.stdout: "chunk_busy": 0, 2026-02-17T12:39:50.394 INFO:tasks.workunit.client.0.trial128.stdout: "locked_object": 0, 2026-02-17T12:39:50.394 INFO:tasks.workunit.client.0.trial128.stdout: "write_blocked_by_scrub": 0, 2026-02-17T12:39:50.394 INFO:tasks.workunit.client.0.trial128.stdout: "scrub_reservations_completed": 0, 2026-02-17T12:39:50.395 INFO:tasks.workunit.client.0.trial128.stdout: "successful_reservations_elapsed": { 2026-02-17T12:39:50.395 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.395 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.395 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.395 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.396 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_aborted": 0, 2026-02-17T12:39:50.396 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_failure": 0, 2026-02-17T12:39:50.396 INFO:tasks.workunit.client.0.trial128.stdout: "reservation_process_skipped": 0, 2026-02-17T12:39:50.396 INFO:tasks.workunit.client.0.trial128.stdout: "failed_reservations_elapsed": { 2026-02-17T12:39:50.396 INFO:tasks.workunit.client.0.trial128.stdout: "avgcount": 0, 2026-02-17T12:39:50.396 INFO:tasks.workunit.client.0.trial128.stdout: "sum": 0, 2026-02-17T12:39:50.396 INFO:tasks.workunit.client.0.trial128.stdout: "avgtime": 0 2026-02-17T12:39:50.396 INFO:tasks.workunit.client.0.trial128.stdout: }, 2026-02-17T12:39:50.396 INFO:tasks.workunit.client.0.trial128.stdout: "replicas_in_reservation": 0 2026-02-17T12:39:50.396 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.396 INFO:tasks.workunit.client.0.trial128.stdout: } 2026-02-17T12:39:50.396 INFO:tasks.workunit.client.0.trial128.stdout: ] 2026-02-17T12:39:50.397 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:39:50.397 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:36: run: echo '-------------- Teardown Test TEST_scrub_test ------------------' 2026-02-17T12:39:50.397 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Teardown Test TEST_scrub_test ------------------ 2026-02-17T12:39:50.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:37: run: teardown td/osd-scrub-test 2026-02-17T12:39:50.398 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:39:50.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:39:50.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:39:50.399 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:39:50.399 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:39:50.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:39:50.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:39:50.399 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:39:50.500 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:39:50.500 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:39:50.501 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:39:50.501 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:39:50.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:39:50.502 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:39:50.503 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:39:50.503 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:39:50.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:39:50.504 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:39:50.504 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:39:50.505 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:39:50.506 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:39:50.507 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:39:50.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:39:50.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:39:50.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:39:50.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:39:50.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:39:50.520 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:39:50.520 INFO:tasks.workunit.client.0.trial128.stdout:-------------- Complete Test TEST_scrub_test ------------------ 2026-02-17T12:39:50.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-test.sh:38: run: echo '-------------- Complete Test TEST_scrub_test ------------------' 2026-02-17T12:39:50.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T12:39:50.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-scrub-test 0 2026-02-17T12:39:50.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-test 2026-02-17T12:39:50.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T12:39:50.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-test KILL 2026-02-17T12:39:50.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:39:50.522 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:39:50.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:39:50.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:39:50.522 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:39:50.523 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:39:50.523 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:39:50.524 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:39:50.524 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:39:50.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:39:50.526 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:39:50.526 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:39:50.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:39:50.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:39:50.527 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:39:50.528 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:39:50.529 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:39:50.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T12:39:50.530 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-test 2026-02-17T12:39:50.531 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:39:50.531 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:39:50.531 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.508089 2026-02-17T12:39:50.532 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.508089 2026-02-17T12:39:50.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:39:50.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:39:50.533 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T12:39:50.534 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T12:39:50.534 DEBUG:teuthology.orchestra.run.trial128:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T12:39:50.586 INFO:tasks.workunit:Running workunit scrub/osd-unexpected-clone.sh... 2026-02-17T12:39:50.586 DEBUG:teuthology.orchestra.run.trial128:workunit test scrub/osd-unexpected-clone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3c96dfb3f50b3d02598a82af7104764bfdcd3919 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh 2026-02-17T12:39:50.639 INFO:tasks.workunit.client.0.trial128.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-17T12:39:50.642 INFO:tasks.workunit.client.0.trial128.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-17T12:39:50.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T12:39:50.642 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-17T12:39:50.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-17T12:39:50.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-17T12:39:50.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-17T12:39:50.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-17T12:39:50.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-17T12:39:50.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-17T12:39:50.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-unexpected-clone 2026-02-17T12:39:50.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:22: run: local dir=td/osd-unexpected-clone 2026-02-17T12:39:50.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:23: run: shift 2026-02-17T12:39:50.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:25: run: export CEPH_MON=127.0.0.1:7144 2026-02-17T12:39:50.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:25: run: CEPH_MON=127.0.0.1:7144 2026-02-17T12:39:50.643 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:26: run: export CEPH_ARGS 2026-02-17T12:39:50.644 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:27: run: uuidgen 2026-02-17T12:39:50.644 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:27: run: CEPH_ARGS+='--fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none ' 2026-02-17T12:39:50.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7144 ' 2026-02-17T12:39:50.645 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:30: run: export -n CEPH_CLI_TEST_DUP_COMMAND 2026-02-17T12:39:50.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:31: run: set 2026-02-17T12:39:50.645 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-17T12:39:50.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:31: run: local funcs=TEST_recover_unexpected 2026-02-17T12:39:50.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:32: run: for func in $funcs 2026-02-17T12:39:50.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:33: run: setup td/osd-unexpected-clone 2026-02-17T12:39:50.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-unexpected-clone 2026-02-17T12:39:50.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-unexpected-clone 2026-02-17T12:39:50.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-unexpected-clone 2026-02-17T12:39:50.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:39:50.647 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-unexpected-clone KILL 2026-02-17T12:39:50.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:39:50.648 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:39:50.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:39:50.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:39:50.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:39:50.648 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:39:50.649 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:39:50.650 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:39:50.650 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:39:50.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:39:50.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:39:50.651 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:39:50.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:39:50.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:39:50.653 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:39:50.653 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:39:50.654 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:39:50.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:39:50.655 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-unexpected-clone 2026-02-17T12:39:50.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:39:50.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:39:50.657 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648935 2026-02-17T12:39:50.657 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.648935 2026-02-17T12:39:50.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:39:50.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:39:50.658 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-unexpected-clone 2026-02-17T12:39:50.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-17T12:39:50.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:39:50.660 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648935 2026-02-17T12:39:50.660 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.648935 2026-02-17T12:39:50.662 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-17T12:39:50.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-17T12:39:50.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-17T12:39:50.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-17T12:39:50.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-unexpected-clone 1' TERM HUP INT 2026-02-17T12:39:50.663 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:34: run: TEST_recover_unexpected td/osd-unexpected-clone 2026-02-17T12:39:50.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:40: TEST_recover_unexpected: local dir=td/osd-unexpected-clone 2026-02-17T12:39:50.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:42: TEST_recover_unexpected: run_mon td/osd-unexpected-clone a 2026-02-17T12:39:50.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-unexpected-clone 2026-02-17T12:39:50.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-17T12:39:50.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-17T12:39:50.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-17T12:39:50.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-unexpected-clone/a 2026-02-17T12:39:50.664 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-unexpected-clone/a --run-dir=td/osd-unexpected-clone 2026-02-17T12:39:50.683 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-17T12:39:50.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:39:50.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:39:50.684 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:39:50.684 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:39:50.684 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648935 2026-02-17T12:39:50.684 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:39:50.685 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-unexpected-clone/a '--log-file=td/osd-unexpected-clone/$name.log' '--admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' --mon-cluster-log-file=td/osd-unexpected-clone/log --run-dir=td/osd-unexpected-clone '--pid-file=td/osd-unexpected-clone/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-17T12:39:50.711 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-17T12:39:50.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-17T12:39:50.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:39:50.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:39:50.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-17T12:39:50.713 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-17T12:39:50.713 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:39:50.713 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:39:50.714 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:39:50.714 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:39:50.714 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:39:50.714 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648935 2026-02-17T12:39:50.714 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.648935/ceph-mon.a.asok 2026-02-17T12:39:50.715 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:39:50.715 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.648935/ceph-mon.a.asok config get fsid 2026-02-17T12:39:50.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-17T12:39:50.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-17T12:39:50.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-17T12:39:50.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-17T12:39:50.770 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-17T12:39:50.771 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-17T12:39:50.771 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-17T12:39:50.771 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-17T12:39:50.772 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-17T12:39:50.772 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:39:50.772 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648935 2026-02-17T12:39:50.772 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.648935/ceph-mon.a.asok 2026-02-17T12:39:50.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-17T12:39:50.772 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.648935/ceph-mon.a.asok config get mon_host 2026-02-17T12:39:50.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:43: TEST_recover_unexpected: run_mgr td/osd-unexpected-clone x 2026-02-17T12:39:50.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-unexpected-clone 2026-02-17T12:39:50.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-17T12:39:50.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-17T12:39:50.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-17T12:39:50.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-unexpected-clone/x 2026-02-17T12:39:50.826 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-17T12:39:50.920 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-17T12:39:50.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:39:50.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:39:50.921 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:39:50.921 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:39:50.921 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648935 2026-02-17T12:39:50.921 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:39:50.922 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:39:50.922 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-unexpected-clone/x '--log-file=td/osd-unexpected-clone/$name.log' '--admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' --run-dir=td/osd-unexpected-clone '--pid-file=td/osd-unexpected-clone/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-17T12:39:50.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:44: TEST_recover_unexpected: run_osd td/osd-unexpected-clone 0 2026-02-17T12:39:50.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-unexpected-clone 2026-02-17T12:39:50.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:39:50.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-17T12:39:50.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:39:50.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-unexpected-clone/0 2026-02-17T12:39:50.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-17T12:39:50.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:39:50.942 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:39:50.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:39:50.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/0' 2026-02-17T12:39:50.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/0/journal' 2026-02-17T12:39:50.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:39:50.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:39:50.943 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-17T12:39:50.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:39:50.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:39:50.943 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:39:50.943 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:39:50.943 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:39:50.943 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648935 2026-02-17T12:39:50.944 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:39:50.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:39:50.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:39:50.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:39:50.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:39:50.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-17T12:39:50.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-17T12:39:50.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:39:50.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:39:50.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:39:50.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:39:50.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:39:50.944 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:39:50.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-unexpected-clone/0 2026-02-17T12:39:50.945 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:39:50.945 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b87b3b16-d3c2-417f-ab7c-7059ca294327 2026-02-17T12:39:50.946 INFO:tasks.workunit.client.0.trial128.stdout:add osd0 b87b3b16-d3c2-417f-ab7c-7059ca294327 2026-02-17T12:39:50.946 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 b87b3b16-d3c2-417f-ab7c-7059ca294327' 2026-02-17T12:39:50.946 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:39:50.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCWYZRpLpqQOBAAdRWJrX8N/lLq4DAy5kbbBg== 2026-02-17T12:39:50.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCWYZRpLpqQOBAAdRWJrX8N/lLq4DAy5kbbBg=="}' 2026-02-17T12:39:50.954 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b87b3b16-d3c2-417f-ab7c-7059ca294327 -i td/osd-unexpected-clone/0/new.json 2026-02-17T12:39:51.040 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:39:51.048 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-unexpected-clone/0/new.json 2026-02-17T12:39:51.050 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/0 --osd-journal=td/osd-unexpected-clone/0/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCWYZRpLpqQOBAAdRWJrX8N/lLq4DAy5kbbBg== --osd-uuid b87b3b16-d3c2-417f-ab7c-7059ca294327 2026-02-17T12:39:51.065 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:51.056+0000 7f5eba7f18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:39:51.066 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:51.060+0000 7f5eba7f18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:39:51.067 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:51.060+0000 7f5eba7f18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:39:51.067 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:51.060+0000 7f5eba7f18c0 -1 bdev(0x55bfc0951c00 td/osd-unexpected-clone/0/block) open stat got: (1) Operation not permitted 2026-02-17T12:39:51.067 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:51.060+0000 7f5eba7f18c0 -1 bluestore(td/osd-unexpected-clone/0) _read_fsid unparsable uuid 2026-02-17T12:39:53.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-unexpected-clone/0/keyring 2026-02-17T12:39:53.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:39:53.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-17T12:39:53.940 INFO:tasks.workunit.client.0.trial128.stdout:adding osd0 key to auth repository 2026-02-17T12:39:53.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-unexpected-clone/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:39:54.131 INFO:tasks.workunit.client.0.trial128.stdout:start osd.0 2026-02-17T12:39:54.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-17T12:39:54.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/0 --osd-journal=td/osd-unexpected-clone/0/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:39:54.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:39:54.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:39:54.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:39:54.151 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:54.144+0000 7ff2757388c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:39:54.153 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:54.144+0000 7ff2757388c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:39:54.155 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:54.148+0000 7ff2757388c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:39:54.281 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-17T12:39:54.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:39:54.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-17T12:39:54.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:39:54.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:39:54.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:39:54.282 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:39:54.283 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:39:54.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:54.284 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:39:54.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:39:55.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:39:55.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:39:55.436 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:39:55.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:39:55.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:55.437 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:39:55.589 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:39:55.651 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:55.644+0000 7ff2757388c0 -1 Falling back to public interface 2026-02-17T12:39:56.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:39:56.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:39:56.590 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:39:56.591 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:39:56.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:56.591 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:39:56.745 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:39:57.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:39:57.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:39:57.746 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:39:57.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:39:57.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:57.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:39:57.900 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:39:58.673 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:39:58.664+0000 7ff2757388c0 -1 osd.0 0 log_to_monitors true 2026-02-17T12:39:58.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:39:58.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:39:58.902 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:39:58.903 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:39:58.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:39:58.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:39:59.071 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:00.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:00.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:00.073 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:40:00.073 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:40:00.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-17T12:40:00.074 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:00.228 INFO:tasks.workunit.client.0.trial128.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/713655167,v1:127.0.0.1:6803/713655167] [v2:127.0.0.1:6804/713655167,v1:127.0.0.1:6805/713655167] exists,up b87b3b16-d3c2-417f-ab7c-7059ca294327 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:45: TEST_recover_unexpected: run_osd td/osd-unexpected-clone 1 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-unexpected-clone 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-unexpected-clone/1 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/1' 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/1/journal' 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-17T12:40:00.229 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:40:00.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:40:00.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:40:00.230 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:40:00.230 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:40:00.230 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648935 2026-02-17T12:40:00.230 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:40:00.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:40:00.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:40:00.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:40:00.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:40:00.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-17T12:40:00.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-17T12:40:00.230 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:40:00.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:40:00.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:40:00.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:40:00.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:40:00.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:40:00.231 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-unexpected-clone/1 2026-02-17T12:40:00.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:40:00.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=22f00513-ced1-4f6e-a9a3-28103f91ed62 2026-02-17T12:40:00.232 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 22f00513-ced1-4f6e-a9a3-28103f91ed62' 2026-02-17T12:40:00.232 INFO:tasks.workunit.client.0.trial128.stdout:add osd1 22f00513-ced1-4f6e-a9a3-28103f91ed62 2026-02-17T12:40:00.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:40:00.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCgYZRpe0pEDhAAhcUv0yzy5plJKpxxBJCQQQ== 2026-02-17T12:40:00.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCgYZRpe0pEDhAAhcUv0yzy5plJKpxxBJCQQQ=="}' 2026-02-17T12:40:00.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 22f00513-ced1-4f6e-a9a3-28103f91ed62 -i td/osd-unexpected-clone/1/new.json 2026-02-17T12:40:00.402 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:40:00.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-unexpected-clone/1/new.json 2026-02-17T12:40:00.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/1 --osd-journal=td/osd-unexpected-clone/1/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCgYZRpe0pEDhAAhcUv0yzy5plJKpxxBJCQQQ== --osd-uuid 22f00513-ced1-4f6e-a9a3-28103f91ed62 2026-02-17T12:40:00.427 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:00.420+0000 7f2555c998c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:00.429 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:00.420+0000 7f2555c998c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:00.430 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:00.424+0000 7f2555c998c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:00.430 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:00.424+0000 7f2555c998c0 -1 bdev(0x55e7ab971c00 td/osd-unexpected-clone/1/block) open stat got: (1) Operation not permitted 2026-02-17T12:40:00.430 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:00.424+0000 7f2555c998c0 -1 bluestore(td/osd-unexpected-clone/1) _read_fsid unparsable uuid 2026-02-17T12:40:02.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-unexpected-clone/1/keyring 2026-02-17T12:40:02.695 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:40:02.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-17T12:40:02.696 INFO:tasks.workunit.client.0.trial128.stdout:adding osd1 key to auth repository 2026-02-17T12:40:02.696 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-unexpected-clone/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:40:02.892 INFO:tasks.workunit.client.0.trial128.stdout:start osd.1 2026-02-17T12:40:02.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-17T12:40:02.893 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/1 --osd-journal=td/osd-unexpected-clone/1/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:40:02.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:40:02.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:40:02.894 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:40:02.913 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:02.904+0000 7efe673738c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:02.915 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:02.908+0000 7efe673738c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:02.916 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:02.908+0000 7efe673738c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:03.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-17T12:40:03.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:40:03.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:40:03.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:40:03.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:40:03.044 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:03.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:40:03.045 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:40:03.045 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:03.046 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:40:03.200 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:04.201 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:04.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:04.202 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:40:04.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:40:04.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:04.202 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:40:04.354 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:04.423 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:04.416+0000 7efe673738c0 -1 Falling back to public interface 2026-02-17T12:40:05.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:05.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:05.355 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:40:05.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:40:05.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:05.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:40:05.509 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:06.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:06.511 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:06.512 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:40:06.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:40:06.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:06.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:40:06.665 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:07.405 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:07.396+0000 7efe673738c0 -1 osd.1 0 log_to_monitors true 2026-02-17T12:40:07.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:07.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:07.667 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:40:07.668 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:40:07.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:07.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:40:07.819 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:08.820 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:08.821 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:08.821 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:40:08.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:40:08.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:08.822 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:40:08.974 INFO:tasks.workunit.client.0.trial128.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4053529278,v1:127.0.0.1:6811/4053529278] [v2:127.0.0.1:6812/4053529278,v1:127.0.0.1:6813/4053529278] exists,up 22f00513-ced1-4f6e-a9a3-28103f91ed62 2026-02-17T12:40:08.977 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:40:08.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:40:08.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:40:08.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:46: TEST_recover_unexpected: run_osd td/osd-unexpected-clone 2 2026-02-17T12:40:08.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-unexpected-clone 2026-02-17T12:40:08.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-17T12:40:08.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-17T12:40:08.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-17T12:40:08.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-unexpected-clone/2 2026-02-17T12:40:08.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-17T12:40:08.978 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:40:08.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:40:08.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:40:08.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/2' 2026-02-17T12:40:08.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/2/journal' 2026-02-17T12:40:08.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-17T12:40:08.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-17T12:40:08.979 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-17T12:40:08.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-17T12:40:08.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:40:08.979 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:40:08.979 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:40:08.979 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:40:08.980 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648935 2026-02-17T12:40:08.980 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:40:08.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:40:08.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:40:08.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-17T12:40:08.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-17T12:40:08.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-17T12:40:08.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-17T12:40:08.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:40:08.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:40:08.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:40:08.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:40:08.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-17T12:40:08.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-17T12:40:08.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-unexpected-clone/2 2026-02-17T12:40:08.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-17T12:40:08.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=d8507b53-2105-4c98-97fd-d33c7128f828 2026-02-17T12:40:08.982 INFO:tasks.workunit.client.0.trial128.stdout:add osd2 d8507b53-2105-4c98-97fd-d33c7128f828 2026-02-17T12:40:08.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 d8507b53-2105-4c98-97fd-d33c7128f828' 2026-02-17T12:40:08.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-17T12:40:08.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCoYZRpKGC6OhAAh/xDx6EqBeepwlq5In0hFw== 2026-02-17T12:40:08.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCoYZRpKGC6OhAAh/xDx6EqBeepwlq5In0hFw=="}' 2026-02-17T12:40:08.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new d8507b53-2105-4c98-97fd-d33c7128f828 -i td/osd-unexpected-clone/2/new.json 2026-02-17T12:40:09.141 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:40:09.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-unexpected-clone/2/new.json 2026-02-17T12:40:09.151 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/2 --osd-journal=td/osd-unexpected-clone/2/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCoYZRpKGC6OhAAh/xDx6EqBeepwlq5In0hFw== --osd-uuid d8507b53-2105-4c98-97fd-d33c7128f828 2026-02-17T12:40:09.166 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:09.160+0000 7f13d4b128c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:09.168 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:09.160+0000 7f13d4b128c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:09.169 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:09.160+0000 7f13d4b128c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:09.170 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:09.164+0000 7f13d4b128c0 -1 bdev(0x56002f5a9c00 td/osd-unexpected-clone/2/block) open stat got: (1) Operation not permitted 2026-02-17T12:40:09.170 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:09.164+0000 7f13d4b128c0 -1 bluestore(td/osd-unexpected-clone/2) _read_fsid unparsable uuid 2026-02-17T12:40:11.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-unexpected-clone/2/keyring 2026-02-17T12:40:11.479 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-17T12:40:11.481 INFO:tasks.workunit.client.0.trial128.stdout:adding osd2 key to auth repository 2026-02-17T12:40:11.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-17T12:40:11.481 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-unexpected-clone/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-17T12:40:11.677 INFO:tasks.workunit.client.0.trial128.stdout:start osd.2 2026-02-17T12:40:11.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-17T12:40:11.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/2 --osd-journal=td/osd-unexpected-clone/2/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:40:11.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-17T12:40:11.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-17T12:40:11.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-17T12:40:11.698 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:11.692+0000 7f561a4f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:11.700 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:11.692+0000 7f561a4f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:11.701 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:11.692+0000 7f561a4f38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:11.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-17T12:40:11.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:40:11.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:40:11.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:40:11.831 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:40:11.832 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:11.832 INFO:tasks.workunit.client.0.trial128.stdout:0 2026-02-17T12:40:11.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:40:11.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:11.833 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:40:11.987 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:12.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:12.989 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:12.989 INFO:tasks.workunit.client.0.trial128.stdout:1 2026-02-17T12:40:12.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:40:12.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:12.990 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:40:13.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:13.195 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:13.188+0000 7f561a4f38c0 -1 Falling back to public interface 2026-02-17T12:40:14.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:14.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:14.145 INFO:tasks.workunit.client.0.trial128.stdout:2 2026-02-17T12:40:14.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:40:14.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:14.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:40:14.300 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:15.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:15.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:15.301 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:40:15.302 INFO:tasks.workunit.client.0.trial128.stdout:3 2026-02-17T12:40:15.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:15.302 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:40:15.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:16.159 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:16.152+0000 7f561a4f38c0 -1 osd.2 0 log_to_monitors true 2026-02-17T12:40:16.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:16.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:16.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:40:16.456 INFO:tasks.workunit.client.0.trial128.stdout:4 2026-02-17T12:40:16.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:16.457 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:40:16.604 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:17.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:17.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:17.605 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-17T12:40:17.606 INFO:tasks.workunit.client.0.trial128.stdout:5 2026-02-17T12:40:17.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:17.606 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:40:17.759 INFO:tasks.workunit.client.0.trial128.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1203401148,v1:127.0.0.1:6819/1203401148] [v2:127.0.0.1:6820/1203401148,v1:127.0.0.1:6821/1203401148] exists,up d8507b53-2105-4c98-97fd-d33c7128f828 2026-02-17T12:40:17.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:40:17.759 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:40:17.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:40:17.760 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:48: TEST_recover_unexpected: ceph osd pool create foo 1 2026-02-17T12:40:17.957 INFO:tasks.workunit.client.0.trial128.stderr:pool 'foo' created 2026-02-17T12:40:17.967 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:49: TEST_recover_unexpected: rados -p foo put foo /etc/passwd 2026-02-17T12:40:18.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:50: TEST_recover_unexpected: rados -p foo mksnap snap 2026-02-17T12:40:18.439 INFO:tasks.workunit.client.0.trial128.stdout:created pool foo snap snap 2026-02-17T12:40:18.441 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:51: TEST_recover_unexpected: rados -p foo put foo /etc/group 2026-02-17T12:40:18.467 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:53: TEST_recover_unexpected: wait_for_clean 2026-02-17T12:40:18.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:40:18.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:40:18.468 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:40:18.468 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:40:18.469 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:40:18.469 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:40:18.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:40:18.469 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:40:18.470 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:40:18.568 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:40:18.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:40:18.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:40:18.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:40:18.569 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:40:18.569 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:40:18.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:40:18.719 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:40:18.719 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:40:18.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:40:18.719 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:40:18.720 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:40:18.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836485 2026-02-17T12:40:18.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836485 2026-02-17T12:40:18.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485' 2026-02-17T12:40:18.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:40:18.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:40:18.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705668 2026-02-17T12:40:18.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705668 2026-02-17T12:40:18.862 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705668' 2026-02-17T12:40:18.863 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:40:18.863 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:40:18.934 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542146 2026-02-17T12:40:18.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542146 2026-02-17T12:40:18.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836485 1-38654705668 2-60129542146' 2026-02-17T12:40:18.935 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:40:18.935 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836485 2026-02-17T12:40:18.936 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:40:18.936 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:40:18.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836485 2026-02-17T12:40:18.937 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:40:18.938 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836485 2026-02-17T12:40:18.939 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.0 seq 21474836485 2026-02-17T12:40:18.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-17T12:40:18.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:40:19.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836485 2026-02-17T12:40:19.091 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:40:20.092 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:40:20.093 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:40:20.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-17T12:40:20.244 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:40:20.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705668 2026-02-17T12:40:20.244 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:40:20.245 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:40:20.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705668 2026-02-17T12:40:20.246 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:40:20.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705668 2026-02-17T12:40:20.247 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-17T12:40:20.248 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.1 seq 38654705668 2026-02-17T12:40:20.248 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:40:20.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-17T12:40:20.412 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:40:20.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542146 2026-02-17T12:40:20.412 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:40:20.413 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:40:20.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542146 2026-02-17T12:40:20.414 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:40:20.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542146 2026-02-17T12:40:20.416 INFO:tasks.workunit.client.0.trial128.stdout:waiting osd.2 seq 60129542146 2026-02-17T12:40:20.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542146' 2026-02-17T12:40:20.417 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:40:20.577 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542146 -lt 60129542146 2026-02-17T12:40:20.577 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:40:20.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:40:20.578 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:40:20.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:40:20.757 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:40:20.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:40:20.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:40:20.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:40:20.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:40:20.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:40:20.758 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:40:20.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:40:20.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:40:20.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:40:20.905 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:40:21.080 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:40:21.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:40:21.081 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:40:21.081 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:55: TEST_recover_unexpected: get_primary foo foo 2026-02-17T12:40:21.081 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1196: get_primary: local poolname=foo 2026-02-17T12:40:21.081 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: get_primary: local objectname=foo 2026-02-17T12:40:21.082 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: get_primary: ceph --format json osd map foo foo 2026-02-17T12:40:21.082 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: get_primary: jq .acting_primary 2026-02-17T12:40:21.229 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:55: TEST_recover_unexpected: local osd=1 2026-02-17T12:40:21.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:57: TEST_recover_unexpected: objectstore_tool td/osd-unexpected-clone 1 --op list foo 2026-02-17T12:40:21.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:57: TEST_recover_unexpected: grep snapid.:1 2026-02-17T12:40:21.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-unexpected-clone 2026-02-17T12:40:21.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:40:21.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:40:21.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:40:21.231 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-unexpected-clone 1 --op list foo 2026-02-17T12:40:21.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-unexpected-clone 2026-02-17T12:40:21.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:40:21.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:40:21.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:40:21.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-unexpected-clone TERM osd.1 2026-02-17T12:40:21.232 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:40:21.232 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:40:21.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:40:21.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:40:21.232 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:40:22.539 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:40:22.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-unexpected-clone 1 --op list foo 2026-02-17T12:40:22.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-unexpected-clone 2026-02-17T12:40:22.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:40:22.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:40:22.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:40:22.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-unexpected-clone/1 2026-02-17T12:40:22.540 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-unexpected-clone/1 --op list foo 2026-02-17T12:40:23.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-unexpected-clone 1 2026-02-17T12:40:23.386 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-unexpected-clone 2026-02-17T12:40:23.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:40:23.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:40:23.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:40:23.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-unexpected-clone/1 2026-02-17T12:40:23.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-17T12:40:23.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:40:23.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:40:23.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:40:23.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/1' 2026-02-17T12:40:23.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/1/journal' 2026-02-17T12:40:23.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:40:23.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:40:23.387 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-17T12:40:23.387 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:40:23.387 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:40:23.387 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:40:23.388 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:40:23.388 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:40:23.388 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648935 2026-02-17T12:40:23.388 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:40:23.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:40:23.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:40:23.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-17T12:40:23.388 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-17T12:40:23.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:40:23.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:40:23.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:40:23.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:40:23.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:40:23.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:40:23.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-unexpected-clone/1 2026-02-17T12:40:23.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:40:23.389 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:40:23.389 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/1 --osd-journal=td/osd-unexpected-clone/1/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:40:23.389 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-unexpected-clone/1/whoami 2026-02-17T12:40:23.390 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:40:23.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:40:23.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:40:23.391 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:40:23.407 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:23.400+0000 7f6215f3c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:23.409 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:23.400+0000 7f6215f3c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:23.410 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:23.404+0000 7f6215f3c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:23.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:40:23.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:40:23.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:40:23.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:40:23.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:40:23.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:23.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:40:23.542 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:40:23.542 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:23.543 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:40:23.699 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:24.671 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:24.664+0000 7f6215f3c8c0 -1 Falling back to public interface 2026-02-17T12:40:24.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:24.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:24.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:40:24.700 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:40:24.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:24.700 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:40:24.857 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:25.858 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:25.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:25.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:40:25.859 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:40:25.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:25.859 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:40:26.012 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:27.014 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:27.014 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:27.014 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:40:27.014 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:40:27.014 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:27.014 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:40:27.168 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:27.694 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:27.684+0000 7f6215f3c8c0 -1 osd.1 17 log_to_monitors true 2026-02-17T12:40:28.170 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:28.170 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:28.170 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:40:28.170 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:40:28.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:28.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:40:28.322 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 21 up_thru 21 down_at 18 last_clean_interval [9,17) [v2:127.0.0.1:6810/3262925948,v1:127.0.0.1:6811/3262925948] [v2:127.0.0.1:6812/3262925948,v1:127.0.0.1:6813/3262925948] exists,up 22f00513-ced1-4f6e-a9a3-28103f91ed62 2026-02-17T12:40:28.322 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:40:28.322 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:40:28.322 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:40:28.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:40:28.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:40:28.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:40:28.323 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:40:28.323 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:40:28.324 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:40:28.324 INFO:tasks.workunit.client.0.trial128.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:40:28.324 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:40:28.324 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:40:28.324 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:40:28.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:40:28.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:40:28.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:40:28.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:40:28.416 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:40:28.416 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:40:28.571 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:40:28.572 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:40:28.572 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:40:28.572 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:40:28.572 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:40:28.572 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:40:28.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836488 2026-02-17T12:40:28.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836488 2026-02-17T12:40:28.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488' 2026-02-17T12:40:28.643 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:40:28.643 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:40:28.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=90194313218 2026-02-17T12:40:28.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 90194313218 2026-02-17T12:40:28.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-90194313218' 2026-02-17T12:40:28.718 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:40:28.718 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:40:28.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542149 2026-02-17T12:40:28.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542149 2026-02-17T12:40:28.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836488 1-90194313218 2-60129542149' 2026-02-17T12:40:28.787 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:40:28.788 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836488 2026-02-17T12:40:28.788 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:40:28.788 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:40:28.789 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836488 2026-02-17T12:40:28.789 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:40:28.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836488 2026-02-17T12:40:28.791 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-17T12:40:28.791 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836488 2026-02-17T12:40:28.791 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:40:28.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836488 2026-02-17T12:40:28.947 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:40:29.948 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:40:29.948 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:40:30.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-17T12:40:30.105 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:40:30.106 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-90194313218 2026-02-17T12:40:30.106 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:40:30.107 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:40:30.108 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-90194313218 2026-02-17T12:40:30.108 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:40:30.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=90194313218 2026-02-17T12:40:30.109 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 90194313218' 2026-02-17T12:40:30.109 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 90194313218 2026-02-17T12:40:30.110 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:40:30.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 90194313218 -lt 90194313218 2026-02-17T12:40:30.271 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:40:30.272 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542149 2026-02-17T12:40:30.272 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:40:30.273 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:40:30.274 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542149 2026-02-17T12:40:30.274 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:40:30.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542149 2026-02-17T12:40:30.275 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542149' 2026-02-17T12:40:30.275 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542149 2026-02-17T12:40:30.275 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:40:30.438 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542149 -lt 60129542149 2026-02-17T12:40:30.439 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:40:30.439 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:40:30.439 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:40:30.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:40:30.624 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:40:30.625 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:40:30.625 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:40:30.625 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:40:30.625 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:40:30.625 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:40:30.625 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:40:30.775 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:40:30.776 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:40:30.776 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:40:30.776 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:40:30.961 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:40:30.961 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:40:30.961 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:40:30.962 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:57: TEST_recover_unexpected: JSON='["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:40:30.962 INFO:tasks.workunit.client.0.trial128.stdout:JSON is ["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}] 2026-02-17T12:40:30.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:58: TEST_recover_unexpected: echo 'JSON is ["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' 2026-02-17T12:40:30.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:59: TEST_recover_unexpected: rm -f td/osd-unexpected-clone/_ td/osd-unexpected-clone/data 2026-02-17T12:40:30.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:60: TEST_recover_unexpected: objectstore_tool td/osd-unexpected-clone 1 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-attr _ 2026-02-17T12:40:30.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-unexpected-clone 2026-02-17T12:40:30.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:40:30.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=1 2026-02-17T12:40:30.964 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:40:30.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-unexpected-clone 1 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-attr _ 2026-02-17T12:40:30.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-unexpected-clone 2026-02-17T12:40:30.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:40:30.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=1 2026-02-17T12:40:30.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:40:30.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-unexpected-clone TERM osd.1 2026-02-17T12:40:30.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:40:30.965 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:40:30.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:40:30.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:40:30.965 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:40:32.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:40:32.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-unexpected-clone 1 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-attr _ 2026-02-17T12:40:32.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-unexpected-clone 2026-02-17T12:40:32.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:40:32.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-17T12:40:32.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:40:32.274 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-unexpected-clone/1 2026-02-17T12:40:32.275 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-unexpected-clone/1 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-attr _ 2026-02-17T12:40:33.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-unexpected-clone 1 2026-02-17T12:40:33.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-unexpected-clone 2026-02-17T12:40:33.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:40:33.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-17T12:40:33.116 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:40:33.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-unexpected-clone/1 2026-02-17T12:40:33.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-17T12:40:33.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:40:33.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:40:33.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:40:33.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/1' 2026-02-17T12:40:33.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/1/journal' 2026-02-17T12:40:33.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:40:33.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:40:33.117 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-17T12:40:33.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:40:33.117 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:40:33.118 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:40:33.118 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:40:33.118 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:40:33.118 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648935 2026-02-17T12:40:33.118 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:40:33.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:40:33.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:40:33.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-17T12:40:33.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-17T12:40:33.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:40:33.120 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:40:33.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:40:33.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:40:33.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:40:33.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:40:33.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-unexpected-clone/1 2026-02-17T12:40:33.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-17T12:40:33.121 INFO:tasks.workunit.client.0.trial128.stderr:start osd.1 2026-02-17T12:40:33.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/1 --osd-journal=td/osd-unexpected-clone/1/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:40:33.121 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-unexpected-clone/1/whoami 2026-02-17T12:40:33.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-17T12:40:33.121 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:40:33.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:40:33.122 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:40:33.135 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:33.128+0000 7f3724dac8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:33.136 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:33.128+0000 7f3724dac8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:33.137 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:33.128+0000 7f3724dac8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:33.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-17T12:40:33.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:40:33.271 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-17T12:40:33.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:40:33.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:40:33.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:33.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:40:33.272 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:40:33.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:33.272 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:40:33.423 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:34.403 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:34.396+0000 7f3724dac8c0 -1 Falling back to public interface 2026-02-17T12:40:34.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:34.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:34.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:40:34.424 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:40:34.424 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:34.425 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:40:34.576 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:35.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:35.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:35.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:40:35.578 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:40:35.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:35.578 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:40:35.727 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:36.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:36.729 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:36.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:40:36.730 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:40:36.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:36.730 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:40:36.878 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:37.409 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:37.400+0000 7f3724dac8c0 -1 osd.1 22 log_to_monitors true 2026-02-17T12:40:37.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:37.879 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:37.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:40:37.880 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:40:37.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:37.880 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-17T12:40:38.027 INFO:tasks.workunit.client.0.trial128.stderr:osd.1 up in weight 1 up_from 26 up_thru 26 down_at 23 last_clean_interval [21,22) [v2:127.0.0.1:6810/75471525,v1:127.0.0.1:6811/75471525] [v2:127.0.0.1:6812/75471525,v1:127.0.0.1:6813/75471525] exists,up 22f00513-ced1-4f6e-a9a3-28103f91ed62 2026-02-17T12:40:38.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:40:38.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:40:38.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:40:38.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:40:38.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:40:38.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:40:38.028 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:40:38.028 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:40:38.029 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:40:38.029 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:40:38.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:40:38.029 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:40:38.030 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:40:38.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:40:38.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:40:38.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:40:38.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:40:38.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:40:38.123 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:40:38.268 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:40:38.269 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:40:38.269 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:40:38.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:40:38.269 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:40:38.269 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:40:38.340 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836491 2026-02-17T12:40:38.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836491 2026-02-17T12:40:38.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491' 2026-02-17T12:40:38.341 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:40:38.341 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:40:38.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149698 2026-02-17T12:40:38.410 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149698 2026-02-17T12:40:38.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-111669149698' 2026-02-17T12:40:38.411 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:40:38.411 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:40:38.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542152 2026-02-17T12:40:38.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542152 2026-02-17T12:40:38.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-111669149698 2-60129542152' 2026-02-17T12:40:38.482 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:40:38.483 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836491 2026-02-17T12:40:38.483 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:40:38.484 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:40:38.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836491 2026-02-17T12:40:38.485 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:40:38.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836491 2026-02-17T12:40:38.486 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-17T12:40:38.486 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836491 2026-02-17T12:40:38.486 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:40:38.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836491 2026-02-17T12:40:38.637 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:40:39.638 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:40:39.639 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:40:39.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-17T12:40:39.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:40:39.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-111669149698 2026-02-17T12:40:39.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:40:39.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:40:39.795 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-111669149698 2026-02-17T12:40:39.796 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:40:39.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149698 2026-02-17T12:40:39.797 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 111669149698' 2026-02-17T12:40:39.797 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 111669149698 2026-02-17T12:40:39.797 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:40:39.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149698 -lt 111669149698 2026-02-17T12:40:39.957 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:40:39.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542152 2026-02-17T12:40:39.958 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:40:39.959 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:40:39.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542152 2026-02-17T12:40:39.960 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:40:39.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542152 2026-02-17T12:40:39.961 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542152' 2026-02-17T12:40:39.961 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 60129542152 2026-02-17T12:40:39.962 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:40:40.123 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542152 -lt 60129542152 2026-02-17T12:40:40.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:40:40.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:40:40.124 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:40:40.307 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:40:40.308 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:40:40.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:40:40.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:40:40.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:40:40.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:40:40.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:40:40.308 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:40:40.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:40:40.454 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:40:40.455 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:40:40.455 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:40:40.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:40:40.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:40:40.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:40:40.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:61: TEST_recover_unexpected: objectstore_tool td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-bytes td/osd-unexpected-clone/data 2026-02-17T12:40:40.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-unexpected-clone 2026-02-17T12:40:40.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:40:40.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T12:40:40.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:40:40.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-bytes td/osd-unexpected-clone/data 2026-02-17T12:40:40.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-unexpected-clone 2026-02-17T12:40:40.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:40:40.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T12:40:40.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:40:40.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-unexpected-clone TERM osd.2 2026-02-17T12:40:40.632 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:40:40.632 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:40:40.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:40:40.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:40:40.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:40:41.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:40:41.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-bytes td/osd-unexpected-clone/data 2026-02-17T12:40:41.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-unexpected-clone 2026-02-17T12:40:41.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:40:41.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T12:40:41.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:40:41.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-unexpected-clone/2 2026-02-17T12:40:41.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-unexpected-clone/2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' get-bytes td/osd-unexpected-clone/data 2026-02-17T12:40:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-unexpected-clone 2 2026-02-17T12:40:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-unexpected-clone 2026-02-17T12:40:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:40:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:40:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:40:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-unexpected-clone/2 2026-02-17T12:40:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-17T12:40:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:40:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:40:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:40:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/2' 2026-02-17T12:40:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/2/journal' 2026-02-17T12:40:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:40:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:40:42.791 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-17T12:40:42.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:40:42.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:40:42.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:40:42.792 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:40:42.792 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:40:42.792 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648935 2026-02-17T12:40:42.792 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:40:42.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:40:42.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:40:42.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-17T12:40:42.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-17T12:40:42.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:40:42.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:40:42.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:40:42.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:40:42.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:40:42.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:40:42.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-unexpected-clone/2 2026-02-17T12:40:42.793 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:40:42.793 INFO:tasks.workunit.client.0.trial128.stderr:start osd.2 2026-02-17T12:40:42.794 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/2 --osd-journal=td/osd-unexpected-clone/2/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:40:42.794 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-unexpected-clone/2/whoami 2026-02-17T12:40:42.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:40:42.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:40:42.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:40:42.795 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:40:42.811 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:42.803+0000 7f9c0a7c28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:42.813 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:42.803+0000 7f9c0a7c28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:42.814 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:42.807+0000 7f9c0a7c28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:42.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:40:42.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:40:42.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:40:42.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:40:42.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:40:42.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:42.947 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:40:42.947 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:40:42.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:42.948 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:40:43.105 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:44.051 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:44.043+0000 7f9c0a7c28c0 -1 Falling back to public interface 2026-02-17T12:40:44.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:44.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:44.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:40:44.107 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:40:44.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:44.107 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:40:44.261 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:45.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:45.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:45.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:40:45.263 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:40:45.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:45.263 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:40:45.415 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:46.416 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:46.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:46.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:40:46.417 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:40:46.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:46.417 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:40:46.571 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:47.060 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:47.051+0000 7f9c0a7c28c0 -1 osd.2 27 log_to_monitors true 2026-02-17T12:40:47.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:47.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:47.573 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:40:47.573 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:40:47.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:47.574 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:40:47.727 INFO:tasks.workunit.client.0.trial128.stderr:osd.2 up in weight 1 up_from 31 up_thru 0 down_at 28 last_clean_interval [14,27) [v2:127.0.0.1:6818/2734829331,v1:127.0.0.1:6819/2734829331] [v2:127.0.0.1:6820/2734829331,v1:127.0.0.1:6821/2734829331] exists,up d8507b53-2105-4c98-97fd-d33c7128f828 2026-02-17T12:40:47.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:40:47.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:40:47.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:40:47.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:40:47.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:40:47.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:40:47.728 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:40:47.728 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:40:47.729 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:40:47.729 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:40:47.729 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:40:47.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:40:47.730 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:40:47.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:40:47.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:40:47.828 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:40:47.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:40:47.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:40:47.829 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:40:47.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:40:47.988 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:40:47.988 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:40:47.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:40:47.988 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:40:47.988 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:40:48.058 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836494 2026-02-17T12:40:48.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836494 2026-02-17T12:40:48.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836494' 2026-02-17T12:40:48.059 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:40:48.059 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:40:48.131 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149701 2026-02-17T12:40:48.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149701 2026-02-17T12:40:48.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836494 1-111669149701' 2026-02-17T12:40:48.132 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:40:48.132 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:40:48.203 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=133143986178 2026-02-17T12:40:48.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 133143986178 2026-02-17T12:40:48.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836494 1-111669149701 2-133143986178' 2026-02-17T12:40:48.204 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:40:48.204 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836494 2026-02-17T12:40:48.205 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:40:48.205 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:40:48.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836494 2026-02-17T12:40:48.206 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:40:48.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836494 2026-02-17T12:40:48.207 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836494' 2026-02-17T12:40:48.207 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836494 2026-02-17T12:40:48.208 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:40:48.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836493 -lt 21474836494 2026-02-17T12:40:48.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:40:49.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:40:49.364 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:40:49.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836494 -lt 21474836494 2026-02-17T12:40:49.517 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:40:49.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-111669149701 2026-02-17T12:40:49.518 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:40:49.519 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:40:49.520 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-111669149701 2026-02-17T12:40:49.520 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:40:49.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149701 2026-02-17T12:40:49.521 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 111669149701' 2026-02-17T12:40:49.521 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 111669149701 2026-02-17T12:40:49.521 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:40:49.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149701 -lt 111669149701 2026-02-17T12:40:49.676 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:40:49.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-133143986178 2026-02-17T12:40:49.677 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:40:49.678 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:40:49.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-133143986178 2026-02-17T12:40:49.679 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:40:49.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=133143986178 2026-02-17T12:40:49.680 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 133143986178' 2026-02-17T12:40:49.680 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 133143986178 2026-02-17T12:40:49.681 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:40:49.842 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 133143986178 -lt 133143986178 2026-02-17T12:40:49.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:40:49.843 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:40:49.843 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:40:50.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:40:50.022 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:40:50.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:40:50.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:40:50.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:40:50.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:40:50.022 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:40:50.023 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:40:50.177 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:40:50.177 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:40:50.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:40:50.178 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:40:50.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:40:50.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:40:50.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:40:50.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:63: TEST_recover_unexpected: rados -p foo rmsnap snap 2026-02-17T12:40:50.430 INFO:tasks.workunit.client.0.trial128.stdout:removed pool foo snap snap 2026-02-17T12:40:50.432 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:65: TEST_recover_unexpected: sleep 5 2026-02-17T12:40:55.434 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:67: TEST_recover_unexpected: objectstore_tool td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes td/osd-unexpected-clone/data 2026-02-17T12:40:55.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-unexpected-clone 2026-02-17T12:40:55.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:40:55.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T12:40:55.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:40:55.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes td/osd-unexpected-clone/data 2026-02-17T12:40:55.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-unexpected-clone 2026-02-17T12:40:55.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:40:55.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T12:40:55.435 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:40:55.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-unexpected-clone TERM osd.2 2026-02-17T12:40:55.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:40:55.436 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:40:55.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:40:55.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:40:55.436 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:40:56.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:40:56.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes td/osd-unexpected-clone/data 2026-02-17T12:40:56.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-unexpected-clone 2026-02-17T12:40:56.746 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:40:56.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T12:40:56.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:40:56.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-unexpected-clone/2 2026-02-17T12:40:56.747 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-unexpected-clone/2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-bytes td/osd-unexpected-clone/data 2026-02-17T12:40:57.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-unexpected-clone 2 2026-02-17T12:40:57.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-unexpected-clone 2026-02-17T12:40:57.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:40:57.909 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:40:57.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:40:57.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-unexpected-clone/2 2026-02-17T12:40:57.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-17T12:40:57.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:40:57.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:40:57.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:40:57.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/2' 2026-02-17T12:40:57.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/2/journal' 2026-02-17T12:40:57.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:40:57.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:40:57.910 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-17T12:40:57.911 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:40:57.911 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:40:57.911 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:40:57.911 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:40:57.911 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:40:57.911 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648935 2026-02-17T12:40:57.911 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:40:57.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:40:57.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:40:57.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-17T12:40:57.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-17T12:40:57.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:40:57.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:40:57.913 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:40:57.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:40:57.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:40:57.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:40:57.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-unexpected-clone/2 2026-02-17T12:40:57.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:40:57.914 INFO:tasks.workunit.client.0.trial128.stderr:start osd.2 2026-02-17T12:40:57.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/2 --osd-journal=td/osd-unexpected-clone/2/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:40:57.914 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-unexpected-clone/2/whoami 2026-02-17T12:40:57.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:40:57.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:40:57.914 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:40:57.915 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:40:57.929 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:57.919+0000 7f725e99a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:57.930 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:57.923+0000 7f725e99a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:57.931 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:57.923+0000 7f725e99a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:40:58.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:40:58.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:40:58.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:40:58.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:40:58.063 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:40:58.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:58.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:40:58.064 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:40:58.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:58.064 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:40:58.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:59.221 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:40:59.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:40:59.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:40:59.222 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:40:59.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:40:59.222 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:40:59.377 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:40:59.407 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:40:59.399+0000 7f725e99a8c0 -1 Falling back to public interface 2026-02-17T12:41:00.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:41:00.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:41:00.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:41:00.379 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:41:00.379 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:41:00.380 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:41:00.535 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:41:01.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:41:01.536 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:41:01.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:41:01.537 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:41:01.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:41:01.537 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:41:01.689 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:41:02.434 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:41:02.427+0000 7f725e99a8c0 -1 osd.2 34 log_to_monitors true 2026-02-17T12:41:02.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:41:02.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:41:02.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:41:02.691 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:41:02.691 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:41:02.692 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:41:02.840 INFO:tasks.workunit.client.0.trial128.stderr:osd.2 up in weight 1 up_from 39 up_thru 0 down_at 35 last_clean_interval [31,34) [v2:127.0.0.1:6818/2466633233,v1:127.0.0.1:6819/2466633233] [v2:127.0.0.1:6820/2466633233,v1:127.0.0.1:6821/2466633233] exists,up d8507b53-2105-4c98-97fd-d33c7128f828 2026-02-17T12:41:02.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:41:02.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:41:02.840 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:41:02.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:41:02.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:41:02.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:41:02.841 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:41:02.841 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:41:02.842 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:41:02.842 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:41:02.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:41:02.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:41:02.842 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:41:02.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:41:02.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:41:02.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:41:02.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:41:02.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:41:02.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:41:03.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:41:03.101 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:41:03.101 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:41:03.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:41:03.101 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:41:03.101 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:41:03.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836498 2026-02-17T12:41:03.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836498 2026-02-17T12:41:03.170 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498' 2026-02-17T12:41:03.171 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:41:03.171 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:41:03.242 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149705 2026-02-17T12:41:03.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149705 2026-02-17T12:41:03.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-111669149705' 2026-02-17T12:41:03.243 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:41:03.243 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:41:03.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724546 2026-02-17T12:41:03.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724546 2026-02-17T12:41:03.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836498 1-111669149705 2-167503724546' 2026-02-17T12:41:03.311 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:41:03.311 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836498 2026-02-17T12:41:03.312 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:41:03.313 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:41:03.314 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836498 2026-02-17T12:41:03.314 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:41:03.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836498 2026-02-17T12:41:03.315 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836498' 2026-02-17T12:41:03.315 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836498 2026-02-17T12:41:03.315 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:41:03.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836498 -lt 21474836498 2026-02-17T12:41:03.470 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:41:03.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-111669149705 2026-02-17T12:41:03.471 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:41:03.472 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:41:03.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-111669149705 2026-02-17T12:41:03.473 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:41:03.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149705 2026-02-17T12:41:03.474 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 111669149705' 2026-02-17T12:41:03.474 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 111669149705 2026-02-17T12:41:03.474 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:41:03.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149704 -lt 111669149705 2026-02-17T12:41:03.628 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:41:04.630 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:41:04.630 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:41:04.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149704 -lt 111669149705 2026-02-17T12:41:04.780 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:41:05.781 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-17T12:41:05.781 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:41:05.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149705 -lt 111669149705 2026-02-17T12:41:05.937 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:41:05.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-167503724546 2026-02-17T12:41:05.938 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:41:05.939 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:41:05.939 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-167503724546 2026-02-17T12:41:05.940 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:41:05.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724546 2026-02-17T12:41:05.941 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 167503724546' 2026-02-17T12:41:05.941 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 167503724546 2026-02-17T12:41:05.941 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:41:06.102 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724546 -lt 167503724546 2026-02-17T12:41:06.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:41:06.103 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:41:06.104 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:41:06.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:41:06.288 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:41:06.289 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:41:06.289 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:41:06.289 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:41:06.289 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:41:06.289 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:41:06.289 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:41:06.443 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:41:06.443 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:41:06.443 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:41:06.444 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:41:06.631 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:41:06.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:41:06.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:41:06.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:68: TEST_recover_unexpected: objectstore_tool td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-attr _ td/osd-unexpected-clone/_ 2026-02-17T12:41:06.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1297: objectstore_tool: local dir=td/osd-unexpected-clone 2026-02-17T12:41:06.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: objectstore_tool: shift 2026-02-17T12:41:06.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1299: objectstore_tool: local id=2 2026-02-17T12:41:06.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: objectstore_tool: shift 2026-02-17T12:41:06.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: objectstore_tool: _objectstore_tool_nowait td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-attr _ td/osd-unexpected-clone/_ 2026-02-17T12:41:06.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1269: _objectstore_tool_nowait: local dir=td/osd-unexpected-clone 2026-02-17T12:41:06.632 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1270: _objectstore_tool_nowait: shift 2026-02-17T12:41:06.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1271: _objectstore_tool_nowait: local id=2 2026-02-17T12:41:06.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1272: _objectstore_tool_nowait: shift 2026-02-17T12:41:06.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1274: _objectstore_tool_nowait: kill_daemons td/osd-unexpected-clone TERM osd.2 2026-02-17T12:41:06.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:41:06.633 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:41:06.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:41:06.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:41:06.633 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:41:07.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:41:07.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1276: _objectstore_tool_nowait: _objectstore_tool_nodown td/osd-unexpected-clone 2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-attr _ td/osd-unexpected-clone/_ 2026-02-17T12:41:07.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/osd-unexpected-clone 2026-02-17T12:41:07.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-17T12:41:07.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-17T12:41:07.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-17T12:41:07.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/osd-unexpected-clone/2 2026-02-17T12:41:07.940 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/osd-unexpected-clone/2 '["1.0",{"oid":"foo","key":"","snapid":1,"hash":2143417350,"max":0,"pool":1,"namespace":"","max":0}]' set-attr _ td/osd-unexpected-clone/_ 2026-02-17T12:41:09.144 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1277: _objectstore_tool_nowait: activate_osd td/osd-unexpected-clone 2 2026-02-17T12:41:09.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-unexpected-clone 2026-02-17T12:41:09.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-17T12:41:09.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-17T12:41:09.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-17T12:41:09.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-unexpected-clone/2 2026-02-17T12:41:09.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 ' 2026-02-17T12:41:09.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-17T12:41:09.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-17T12:41:09.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-17T12:41:09.145 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-unexpected-clone/2' 2026-02-17T12:41:09.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-unexpected-clone/2/journal' 2026-02-17T12:41:09.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-17T12:41:09.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-17T12:41:09.146 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-unexpected-clone' 2026-02-17T12:41:09.146 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-17T12:41:09.146 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-17T12:41:09.146 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-17T12:41:09.146 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-17T12:41:09.146 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:41:09.146 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648935 2026-02-17T12:41:09.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:41:09.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' 2026-02-17T12:41:09.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-17T12:41:09.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-unexpected-clone/$name.log' 2026-02-17T12:41:09.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-unexpected-clone/$name.pid' 2026-02-17T12:41:09.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-17T12:41:09.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-17T12:41:09.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-17T12:41:09.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-17T12:41:09.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-17T12:41:09.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-17T12:41:09.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-unexpected-clone/2 2026-02-17T12:41:09.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-17T12:41:09.149 INFO:tasks.workunit.client.0.trial128.stderr:start osd.2 2026-02-17T12:41:09.149 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=befd0c68-bb71-47f6-9793-e91fc23f40c4 --auth-supported=none --mon-host=127.0.0.1:7144 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-unexpected-clone/2 --osd-journal=td/osd-unexpected-clone/2/journal --chdir= --run-dir=td/osd-unexpected-clone '--admin-socket=/tmp/ceph-asok.648935/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-unexpected-clone/$name.log' '--pid-file=td/osd-unexpected-clone/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-17T12:41:09.149 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-unexpected-clone/2/whoami 2026-02-17T12:41:09.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-17T12:41:09.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-17T12:41:09.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-17T12:41:09.150 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-17T12:41:09.164 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:41:09.155+0000 7f01dff358c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:41:09.166 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:41:09.159+0000 7f01dff358c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:41:09.167 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:41:09.159+0000 7f01dff358c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-17T12:41:09.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-17T12:41:09.304 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-17T12:41:09.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-17T12:41:09.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-17T12:41:09.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-17T12:41:09.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:41:09.305 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-17T12:41:09.305 INFO:tasks.workunit.client.0.trial128.stderr:0 2026-02-17T12:41:09.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:41:09.306 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:41:09.454 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:41:10.408 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:41:10.399+0000 7f01dff358c0 -1 Falling back to public interface 2026-02-17T12:41:10.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:41:10.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:41:10.455 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-17T12:41:10.456 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:41:10.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:41:10.456 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:41:10.601 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:41:11.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:41:11.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:41:11.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-17T12:41:11.651 INFO:tasks.workunit.client.0.trial128.stderr:2 2026-02-17T12:41:11.651 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:41:11.652 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:41:11.752 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:41:12.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:41:12.754 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:41:12.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-17T12:41:12.755 INFO:tasks.workunit.client.0.trial128.stderr:3 2026-02-17T12:41:12.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:41:12.755 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:41:12.903 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-17T12:41:13.398 INFO:tasks.workunit.client.0.trial128.stderr:2026-02-17T12:41:13.391+0000 7f01dff358c0 -1 osd.2 40 log_to_monitors true 2026-02-17T12:41:13.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-17T12:41:13.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-17T12:41:13.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-17T12:41:13.904 INFO:tasks.workunit.client.0.trial128.stderr:4 2026-02-17T12:41:13.904 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-17T12:41:13.905 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-17T12:41:14.051 INFO:tasks.workunit.client.0.trial128.stderr:osd.2 up in weight 1 up_from 44 up_thru 0 down_at 41 last_clean_interval [39,40) [v2:127.0.0.1:6818/3756503102,v1:127.0.0.1:6819/3756503102] [v2:127.0.0.1:6820/3756503102,v1:127.0.0.1:6821/3756503102] exists,up d8507b53-2105-4c98-97fd-d33c7128f828 2026-02-17T12:41:14.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-17T12:41:14.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-17T12:41:14.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-17T12:41:14.051 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: objectstore_tool: wait_for_clean 2026-02-17T12:41:14.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-17T12:41:14.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-17T12:41:14.052 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-17T12:41:14.052 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-17T12:41:14.053 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-17T12:41:14.053 INFO:tasks.workunit.client.0.trial128.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-17T12:41:14.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-17T12:41:14.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-17T12:41:14.053 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-17T12:41:14.147 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-17T12:41:14.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-17T12:41:14.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-17T12:41:14.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-17T12:41:14.148 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-17T12:41:14.148 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-17T12:41:14.294 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-17T12:41:14.294 INFO:tasks.workunit.client.0.trial128.stderr:1 2026-02-17T12:41:14.294 INFO:tasks.workunit.client.0.trial128.stderr:2' 2026-02-17T12:41:14.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-17T12:41:14.295 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:41:14.295 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-17T12:41:14.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836501 2026-02-17T12:41:14.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836501 2026-02-17T12:41:14.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501' 2026-02-17T12:41:14.367 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:41:14.368 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-17T12:41:14.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=111669149708 2026-02-17T12:41:14.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 111669149708 2026-02-17T12:41:14.439 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-111669149708' 2026-02-17T12:41:14.440 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-17T12:41:14.440 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-17T12:41:14.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561026 2026-02-17T12:41:14.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561026 2026-02-17T12:41:14.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836501 1-111669149708 2-188978561026' 2026-02-17T12:41:14.512 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:41:14.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836501 2026-02-17T12:41:14.513 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:41:14.513 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-17T12:41:14.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836501 2026-02-17T12:41:14.514 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:41:14.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836501 2026-02-17T12:41:14.515 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836501' 2026-02-17T12:41:14.516 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.0 seq 21474836501 2026-02-17T12:41:14.516 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:41:14.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836500 -lt 21474836501 2026-02-17T12:41:14.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-17T12:41:15.670 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-17T12:41:15.671 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-17T12:41:15.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836502 -lt 21474836501 2026-02-17T12:41:15.825 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:41:15.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-111669149708 2026-02-17T12:41:15.826 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:41:15.827 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-17T12:41:15.828 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-111669149708 2026-02-17T12:41:15.828 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:41:15.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=111669149708 2026-02-17T12:41:15.829 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 111669149708' 2026-02-17T12:41:15.829 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.1 seq 111669149708 2026-02-17T12:41:15.830 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-17T12:41:15.980 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 111669149708 -lt 111669149708 2026-02-17T12:41:15.981 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-17T12:41:15.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-188978561026 2026-02-17T12:41:15.981 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-17T12:41:15.982 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-17T12:41:15.982 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-188978561026 2026-02-17T12:41:15.983 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-17T12:41:15.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561026 2026-02-17T12:41:15.984 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 188978561026' 2026-02-17T12:41:15.984 INFO:tasks.workunit.client.0.trial128.stderr:waiting osd.2 seq 188978561026 2026-02-17T12:41:15.984 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-17T12:41:16.139 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561026 -lt 188978561026 2026-02-17T12:41:16.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-17T12:41:16.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:41:16.140 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:41:16.327 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-17T12:41:16.328 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-17T12:41:16.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-17T12:41:16.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-17T12:41:16.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-17T12:41:16.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-17T12:41:16.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-17T12:41:16.328 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-17T12:41:16.480 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-17T12:41:16.480 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-17T12:41:16.480 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-17T12:41:16.481 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-17T12:41:16.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-17T12:41:16.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-17T12:41:16.668 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-17T12:41:16.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:70: TEST_recover_unexpected: sleep 5 2026-02-17T12:41:21.669 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:72: TEST_recover_unexpected: ceph pg repair 1.0 2026-02-17T12:41:21.808 INFO:tasks.workunit.client.0.trial128.stderr:instructing pg 1.0 on osd.1 to repair 2026-02-17T12:41:21.817 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:74: TEST_recover_unexpected: sleep 10 2026-02-17T12:41:31.818 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:76: TEST_recover_unexpected: ceph log last 2026-02-17T12:41:31.989 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T12:41:23.243703+0000 mon.a (mon.0) 268 : cluster [ERR] Health check failed: Possible data damage: 1 pg inconsistent (PG_DAMAGED) 2026-02-17T12:41:31.989 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T12:41:22.468860+0000 osd.1 (osd.1) 2 : cluster [ERR] 1.0 shard 0 1:602f83fe:::foo:1 : missing 2026-02-17T12:41:31.989 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T12:41:22.468862+0000 osd.1 (osd.1) 3 : cluster [ERR] 1.0 shard 1 1:602f83fe:::foo:1 : missing 2026-02-17T12:41:31.989 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T12:41:22.468940+0000 osd.1 (osd.1) 4 : cluster [ERR] repair 1.0 1:602f83fe:::foo:1 : is an unexpected clone 2026-02-17T12:41:31.989 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T12:41:22.469030+0000 osd.1 (osd.1) 5 : cluster [ERR] 1.0 repair 1 missing, 0 inconsistent objects 2026-02-17T12:41:31.989 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T12:41:22.469040+0000 osd.1 (osd.1) 6 : cluster [ERR] 1.0 repair 1 missing, 0 inconsistent objects 2026-02-17T12:41:31.989 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T12:41:22.469059+0000 osd.1 (osd.1) 7 : cluster [ERR] 1.0 repair 3 errors, 2 fixed 2026-02-17T12:41:31.989 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T12:41:22.473634+0000 osd.1 (osd.1) 9 : cluster [ERR] 1.0 shard 1 soid 1:602f83fe:::foo:1 : candidate size 0 info size 15806 mismatch 2026-02-17T12:41:31.990 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T12:41:22.473682+0000 osd.1 (osd.1) 10 : cluster [ERR] 1.0 shard 0 soid 1:602f83fe:::foo:1 : data_digest 0xdfb728bf != data_digest 0xf85c6ade from auth oi 1:602f83fe:::foo:1(17'2 client.4174.0:1 dirty|data_digest s 15806 uv 1 dd f85c6ade alloc_hint [0 0 0]) 2026-02-17T12:41:31.990 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T12:41:22.473683+0000 osd.1 (osd.1) 11 : cluster [ERR] 1.0 shard 1 soid 1:602f83fe:::foo:1 : data_digest 0xffffffff != data_digest 0xdfb728bf from shard 0, data_digest 0xffffffff != data_digest 0xf85c6ade from auth oi 1:602f83fe:::foo:1(17'2 client.4174.0:1 dirty|data_digest s 15806 uv 1 dd f85c6ade alloc_hint [0 0 0]), size 0 != size 15806 from auth oi 1:602f83fe:::foo:1(17'2 client.4174.0:1 dirty|data_digest s 15806 uv 1 dd f85c6ade alloc_hint [0 0 0]), size 0 != size 15806 from shard 0 2026-02-17T12:41:31.990 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T12:41:22.473683+0000 osd.1 (osd.1) 12 : cluster [ERR] 1.0 soid 1:602f83fe:::foo:1 : data_digest 0xf85c6ade != data_digest 0xdfb728bf from shard 0 2026-02-17T12:41:31.990 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T12:41:22.473746+0000 osd.1 (osd.1) 13 : cluster [ERR] deep-scrub 1.0 1:602f83fe:::foo:1 : is an unexpected clone 2026-02-17T12:41:31.990 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T12:41:22.473854+0000 osd.1 (osd.1) 14 : cluster [ERR] 1.0 deep-scrub 0 missing, 1 inconsistent objects 2026-02-17T12:41:31.990 INFO:tasks.workunit.client.0.trial128.stdout:2026-02-17T12:41:22.473856+0000 osd.1 (osd.1) 15 : cluster [ERR] 1.0 deep-scrub 4 errors 2026-02-17T12:41:31.997 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:79: TEST_recover_unexpected: timeout 60 ceph tell osd.0 version 2026-02-17T12:41:32.065 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:41:32.065 INFO:tasks.workunit.client.0.trial128.stdout: "version": "19.2.3-675-g3c96dfb3", 2026-02-17T12:41:32.065 INFO:tasks.workunit.client.0.trial128.stdout: "release": "squid", 2026-02-17T12:41:32.065 INFO:tasks.workunit.client.0.trial128.stdout: "release_type": "stable" 2026-02-17T12:41:32.066 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:41:32.072 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:80: TEST_recover_unexpected: timeout 60 ceph tell osd.1 version 2026-02-17T12:41:32.136 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:41:32.136 INFO:tasks.workunit.client.0.trial128.stdout: "version": "19.2.3-675-g3c96dfb3", 2026-02-17T12:41:32.136 INFO:tasks.workunit.client.0.trial128.stdout: "release": "squid", 2026-02-17T12:41:32.137 INFO:tasks.workunit.client.0.trial128.stdout: "release_type": "stable" 2026-02-17T12:41:32.137 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:41:32.143 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:81: TEST_recover_unexpected: timeout 60 ceph tell osd.2 version 2026-02-17T12:41:32.209 INFO:tasks.workunit.client.0.trial128.stdout:{ 2026-02-17T12:41:32.209 INFO:tasks.workunit.client.0.trial128.stdout: "version": "19.2.3-675-g3c96dfb3", 2026-02-17T12:41:32.209 INFO:tasks.workunit.client.0.trial128.stdout: "release": "squid", 2026-02-17T12:41:32.209 INFO:tasks.workunit.client.0.trial128.stdout: "release_type": "stable" 2026-02-17T12:41:32.209 INFO:tasks.workunit.client.0.trial128.stdout:} 2026-02-17T12:41:32.218 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-unexpected-clone.sh:35: run: teardown td/osd-unexpected-clone 2026-02-17T12:41:32.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-unexpected-clone 2026-02-17T12:41:32.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-17T12:41:32.219 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-unexpected-clone KILL 2026-02-17T12:41:32.220 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:41:32.220 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:41:32.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:41:32.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:41:32.220 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:41:32.330 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:41:32.330 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:41:32.331 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:41:32.332 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:41:32.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:41:32.333 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:41:32.334 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:41:32.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:41:32.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:41:32.335 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:41:32.335 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:41:32.336 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:41:32.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-17T12:41:32.337 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-unexpected-clone 2026-02-17T12:41:32.349 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:41:32.349 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:41:32.349 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648935 2026-02-17T12:41:32.349 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.648935 2026-02-17T12:41:32.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:41:32.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:41:32.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-17T12:41:32.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-unexpected-clone 0 2026-02-17T12:41:32.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-unexpected-clone 2026-02-17T12:41:32.350 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-17T12:41:32.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-unexpected-clone KILL 2026-02-17T12:41:32.351 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-17T12:41:32.351 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-17T12:41:32.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-17T12:41:32.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-17T12:41:32.351 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-17T12:41:32.353 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-17T12:41:32.354 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-17T12:41:32.355 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-17T12:41:32.355 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-17T12:41:32.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-17T12:41:32.356 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-17T12:41:32.356 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-17T12:41:32.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:41:32.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-17T12:41:32.358 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-17T12:41:32.358 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T12:41:32.359 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:41:32.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-17T12:41:32.361 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-unexpected-clone 2026-02-17T12:41:32.362 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-17T12:41:32.362 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-17T12:41:32.362 INFO:tasks.workunit.client.0.trial128.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.648935 2026-02-17T12:41:32.362 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.648935 2026-02-17T12:41:32.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-17T12:41:32.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-17T12:41:32.363 INFO:tasks.workunit.client.0.trial128.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-17T12:41:32.364 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-17T12:41:32.365 DEBUG:teuthology.orchestra.run.trial128:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-17T12:41:32.418 INFO:tasks.workunit:Stopping ['scrub'] on client.0... 2026-02-17T12:41:32.418 DEBUG:teuthology.orchestra.run.trial128:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-17T12:41:32.763 DEBUG:teuthology.parallel:result is None 2026-02-17T12:41:32.764 DEBUG:teuthology.orchestra.run.trial128:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-17T12:41:32.772 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-17T12:41:32.772 DEBUG:teuthology.orchestra.run.trial128:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-17T12:41:32.817 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-17T12:41:32.817 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-17T12:41:32.826 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-17T12:41:32.827 DEBUG:teuthology.orchestra.run.trial128:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-17T12:41:32.973 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-17T12:41:32.984 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-17T12:41:32.984 DEBUG:teuthology.orchestra.run.trial128:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T12:41:33.040 INFO:teuthology.orchestra.run.trial128.stdout: remote refid st t when poll reach delay offset jitter 2026-02-17T12:41:33.041 INFO:teuthology.orchestra.run.trial128.stdout:============================================================================== 2026-02-17T12:41:33.041 INFO:teuthology.orchestra.run.trial128.stdout:-172-232-15-202. 80.192.165.246 2 u 123 128 377 27.277 +10.242 0.532 2026-02-17T12:41:33.041 INFO:teuthology.orchestra.run.trial128.stdout:+s2-a.time.mci1. 135.45.28.167 2 u 120 128 377 29.773 +3.069 0.345 2026-02-17T12:41:33.041 INFO:teuthology.orchestra.run.trial128.stdout:*ntp2.wiktel.com .GPS. 1 u 70 128 377 37.910 +5.342 0.384 2026-02-17T12:41:33.041 INFO:teuthology.orchestra.run.trial128.stdout:+nyc3.us.ntp.li 17.253.2.35 2 u 126 128 377 3.289 +3.476 2.525 2026-02-17T12:41:33.042 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-17T12:41:33.053 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-17T12:41:33.053 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-17T12:41:33.066 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-17T12:41:33.081 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-17T12:41:33.095 INFO:teuthology.task.internal:Duration was 4738.777821 seconds 2026-02-17T12:41:33.096 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-17T12:41:33.116 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-17T12:41:33.116 DEBUG:teuthology.orchestra.run.trial128:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-17T12:41:33.143 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-17T12:41:33.143 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial128.front.sepia.ceph.com 2026-02-17T12:41:33.143 DEBUG:teuthology.orchestra.run.trial128:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-17T12:41:33.191 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-17T12:41:33.191 DEBUG:teuthology.orchestra.run.trial128:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T12:41:33.391 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-17T12:41:33.391 DEBUG:teuthology.orchestra.run.trial128:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T12:41:33.401 INFO:teuthology.orchestra.run.trial128.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T12:41:33.402 INFO:teuthology.orchestra.run.trial128.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T12:41:33.402 INFO:teuthology.orchestra.run.trial128.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-17T12:41:33.402 INFO:teuthology.orchestra.run.trial128.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T12:41:33.402 INFO:teuthology.orchestra.run.trial128.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-17T12:41:33.448 INFO:teuthology.orchestra.run.trial128.stderr: 81.7% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-17T12:41:33.450 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-17T12:41:33.465 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-17T12:41:33.465 DEBUG:teuthology.orchestra.run.trial128:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-17T12:41:33.502 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-17T12:41:33.518 DEBUG:teuthology.orchestra.run.trial128:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:41:33.553 INFO:teuthology.orchestra.run.trial128.stdout:kernel.core_pattern = core 2026-02-17T12:41:33.561 DEBUG:teuthology.orchestra.run.trial128:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-17T12:41:33.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T12:41:33.605 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-17T12:41:33.621 INFO:teuthology.task.internal:Transferring archived files... 2026-02-17T12:41:33.624 DEBUG:teuthology.misc:Transferring archived files from trial128:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/skanta-2026-02-17_04:10:36-rados-wip-bharath20-testing-2026-02-13-0906-squid-distro-default-trial/51298/remote/trial128 2026-02-17T12:41:33.625 DEBUG:teuthology.orchestra.run.trial128:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-17T12:41:33.678 INFO:teuthology.task.internal:Removing archive directory... 2026-02-17T12:41:33.678 DEBUG:teuthology.orchestra.run.trial128:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-17T12:41:33.710 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-17T12:41:33.728 INFO:teuthology.task.internal:Not uploading archives. 2026-02-17T12:41:33.728 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-17T12:41:33.744 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-17T12:41:33.744 DEBUG:teuthology.orchestra.run.trial128:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-17T12:41:33.761 INFO:teuthology.orchestra.run.trial128.stdout: 1180220 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 12:41 /home/ubuntu/cephtest 2026-02-17T12:41:33.762 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-17T12:41:33.779 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-17T12:41:33.817 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} duration: 4738.777820587158 flavor: default owner: scheduled_skanta@soko04.front.sepia.ceph.com success: true 2026-02-17T12:41:33.817 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T12:41:33.942 INFO:teuthology.run:pass